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Typographic Conventions 


American English is the standard used in this handbook. 


The following typographic conventions are also used. 
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Examine Data Services Objects in Designer 


[y 


. Launch the Data Services Designer. 

. Create a new Project named PR_Demo01. 

. Create a new job, JOB_Demo01 , in the PR_Demo01 project. 
. Create a new Project in the Local Object Library. 

. Create a new Job in the Local Object Library. 

. Add JOB_Demo02 job to the PR_Demo02 project. 

. Make the PR_Demo01 the active project in Designer. 


. Add JOB_Demo02 job to the PR_Demo01 project. 
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. Add a data flow, DF_Demo01 , to the JOB_Demo02 job in the PR_Demo01 project. 


m 
O 


. Open the project PR_Demo02. 


a 
En 


. Inthe Project Area, expand JOB_Demo02 in PR Demo01. 


m 
N 


. Add a Script to the Job_Demo02 job. 


m 
w 


. Discuss the Local Object Library . Click through the tabs and explain the different types of 
objects. Explain that these objects are all stored in the local repository. 


Ke 
P 


. Save and execute the Job_Demo02. 
You can see the print function line in the trace log. 
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Examine Data Services Objects in Designer 


1. Launch the Data Services Designer. 


a) Choose Start — Group D SAP Data Services — Data Services Designer. 
Alternatively, choose Start. The Start menu opens as a set of tiles. 


Choose the white arrow at the bottom of the desktop. This is the equivalent of Start > 
All Programs. 


Scroll to the right until you find the SAP Data Services and choose the Data Services 
Designer tile. 


b) To login, in the SAP Data Services Repository Login dialog box, enter the following: 


c) Choose Log On. 





d) Select Repository DSREPO##. 


2. Create a new Project named PR_Demo01. 


a) In the Designer Workspace, on the Start page, choose Create Project. 


b) Enter the project name PR_Demo01. 


3. Create anew job, JOB_Demo01, in the PR Demo01 project. 


a) Inthe project area, right-click the PR_Demo01 project and choose New Batch Job . 


b) Name the job JOB_Demo01 and, on your keyboard, press the Enter key. 


4. Create a new Project in the Local Object Library. 


a) Inthe Local Object Library , choose the Projects tab. 


b) Inthe white space of the Local Object Library right-click in the white space and choose 
New. 


c) Enter the project name PR_Demo02, and choose Create. 


d) Explain that the new project is now the active project in the Project Area. 


5. Create a new Job in the Local Object Library. 


a) Click on the Jobs tab in the Local Object Library. 
b) Right-click Batch Jobs and choose New . 
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c) Name the new job JOB_Demo02. 


6. Add JOB_Demo02 job to the PR_Demo02 project. 
a) In the Local Object Library , select JOB_Demo02 and drag it to the Project Area . 


b) Explain that a batch job can be created independently of a project, but it must be 
added to a project to be executed. 


7. Make the PR_Demo01 the active project in Designer. 
a) In the Local Object Library , choose the Projects tab. 


b) To make the PR Demo01 project active, double-click it. 
c) Explain that only one project can be active at a time in the Designer. 


8. Add JOB_Demo02 job to the PR Demo01 project. 
a) In the Local Object Library , choose the Jobs tab. 


b) Select JOB_Demo02 and drag it to the Project Area. 


9. Add a data flow, DF_Demo01, tothe JOB_Demo02 job inthe PR Demo01 project. 
a) In the Project Area , select JOB_Demo02 in the Project Area. 


Explain that the job is now open in the workspace and the tool palette has been 
activated. 


b) In the tool palette, choose the Data Flow icon. 

c) To add the data flow to the job, click in the workspace. 
d) Enter the data flow name, DF_Demo01 . 

e) To open DF_Demo01 , double-click it. 


f) Toggle between the JOB_Demo02 tab and the DF_Demo01 tab, to see that different 
objects are active in the tool palette when the data flow is being edited in the 
workspace, compared to the job. 


Note: 
Objects such as scripts and conditionals can be used in a job, but not a 
data flow. 


Objects such as the query transform and template tables can be used ina 
data flow, but not a job. 





10. Open the project PR_Demo02. 
a) In the Local Objects Library, choose the Projects tab. 


b) To make the PR Demo01 project active, double-click it. 
c) Explain that all windows from PR_Demo01 have been closed in the workspace. 


11. Inthe Project Area, expand JOB_Demo02 in PR_Demo01. 


a) Explain that a job is a reusable object. 
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A reusable object has a single definition; all calls to the object refer to that definition. If 
you change the definition of the object in one place, you are changing the object in all 
other places in which it appears. 


12. Adda Script to the Job_Demo02 job. 
a) To add the Script, inthe tool palette, choose the Script icon and click in the workspace 
to the left of the DF_Demo02 data flow. 


b) Connect the Script to the data flow. 
c) To edit the Script, double-click it. 


d) Add a print statement in the script, as follows: 
print(‘This script is run in the job Job Demo02’) ; 


Note: 


Type this expression exactly as it is. 





e) Explain that the Script is a single use object. 


13. Discuss the Local Object Library . Click through the tabs and explain the different types of 
objects. Explain that these objects are all stored in the local repository. 


14. Save and execute the Job_Demo02. 
You can see the print function line in the trace log. 
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Create Source and Target Datastores 


Business Example 


You are working as an ETL developer using SAP Data Services Designer. You need to create 
datastores for the source, target, and staging databases. 


Note: 


When the data values for the exercise include ##, replace ## with the number 
that your instructor has provided to you. 





Task 1: Start the SAP BusinessObjects Data Services Designer 
1. Login to the Data Services Designer. 


Task 2: Create Datastores and Import Metadata for Databases 


Note: 
The Database Name, User name and Password are all case sensitive. 





1. In your Local Object Library, create a new source Datastore for the Alpha Acquisitions 
database. 


Table 1: Alpha Datastore Values 


2. Import the metadata for the Alpha Acquisitions database source tables. 





3. In your Local Object Library, create a new Datastore for the Delta staging database. 
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4. Import the metadata for the Delta database source tables. 





5. In your Local Object Library, create a new target Datastore for the HR Data Mart. 


6. Import the metadata for the HR_datamart database source tables. 





7. In your Local Object Library, create a new target Datastore for the Omega data warehouse. 
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Create Source and Target Datastores 


Business Example 


You are working as an ETL developer using SAP Data Services Designer. You need to create 
datastores for the source, target, and staging databases. 


Note: 


When the data values for the exercise include ##, replace ## with the number 
that your instructor has provided to you. 





Task 1: Start the SAP BusinessObjects Data Services Designer 


1. Login to the Data Services Designer. 


a) In the Windows Terminal Server (WTS) training environment desktops, choose Start 
— All Programs — _SAP Data Services 4.2 — Data Services Designer. 


b) The System-host[:port] field should be: WDFLBMT5074 : 6400 


c) In the SAP Data Services Repository Login dialog box, in the User name field, enter your 
user ID, train-Xx. 


d) In the password field, enter your password, which is the same as your user name. 
e) Choose Log on. 

f) From the list of repositories, choose your repository, DSREPOXX. 

g) Choose OK. 


Task 2: Create Datastores and Import Metadata for Databases 


Note: 


The Database Name, User name and Password are all case sensitive. 





1. In your Local Object Library, create a new source Datastore for the Alpha Acquisitions 
database. 


Table 1: Alpha Datastore Values 


Database type Sybase ASE 





Unit 2: Source and Target Metadata 


WDFLBMT5074 


ALPHA 
sourceuser 
sourcepass 


a) Inthe Local Object Library, choose the Datastores tab. 





b) Right click the white workspace of the tab and choose New. 


c) Inthe resulting dialog box, in the appropriate fields, enter the values from the Alpha 
Datastore Values table. 


d) To save the Datastore, choose OK. 
e) Toclose the display, choose the x icon in the upper right corner of the data display. 


2. Import the metadata for the Alpha Acquisitions database source tables. 
a) Right click the Alpha datastore that you just created and choose Open. 


You will see the following list of tables: 

. dbo.category 

e dbo.city 

. dbo.country 

e dbo.customer 

. dbo.department 

. dbo.employee 

. dbo.hr_comp_update 

e dbo.order_details 

e dbo.orders 

. dbo.product 

. dbo.region 
b) To select all of the tables, hold the CTRL key and click each table name. 
c) Right click the selected tables and choose Import. 
d) To close the view of Alpha tables, on the tool bar, choose Back. 


e) To confirm that there are four records in the Alpha. Category table, right click the 
Category table in Local Object Library Datastore tab and choose View Data. 


f) Close the data display. 


3. In your Local Object Library, create a new Datastore for the Delta staging database. 
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a) In the Local Object Library, choose the Datastores tab. 





b) Right click the white workspace of the tab and choose New. 


c) Inthe resulting dialog box, in the appropriate fields, enter the values from the table, 
above 


d) To save the Datastore, choose OK. 


4. Import the metadata for the Delta database source tables. 


a) Right click the Delta datastore that you just created, and choose Open. 
You will see the following table you need to import: dbo.Employee_Comp. 


b) Right click the table: dbo.Employee_Comp, and choose Import. 
c) On the tool bar, choose Back to close the Delta Tables view. 


5. In your Local Object Library, create a new target Datastore for the HR Data Mart. 





a) Inthe Local Object Library, choose the Datastores tab, right click the white workspace 
of the tab and choose New. 


b) In the resulting dialog box, in the appropriate fields, enter the values from the table 
above. 


c) To save the Datastore, choose Ok. 
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6. Import the metadata for the HR_datamart database source tables. 


a) Right-click the HR_datamart datastore that you have just created. 
b) Choose Import By Name. 


c) Enter the table name EMP_DEPT. 


The owner is dbo. 
d) Repeat steps b and c for the following table names: 
« EMPLOYEE 
+ HR COMP UPDATE 
e RECOVERY STATUS 


e) To close the view of the HR_datamart table, choose Back. 


7. In your Local Object Library, create a new target Datastore for the Omega data warehouse. 


a) In the Local Object Library, choose the Datastores tab, right click the white workspace 
of the tab and choose New. 





b) In the resulting dialog box, enter the values from the table above. 


c) To import the metadata for the Omega database source tables, right click the Omega 
datastore that you just created and choose Open. 
You will see a list of tables: 


. dbo.emp_dim 

e dbo.product_dim 

. dbo.product_target 
« dbo.time_dim 


d) Toselect all tables, select the first table and, while holding down the Shift key on the 
keyboard, select the last table. 


e) Right click the selected tables and choose Import. 
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Solution 2: Create Source and Target Datastores 


f) Close the view of Omega tables. 


g) To save your work, from the main menu, choose Project — Save All. 


Note: 


All table metadata must be imported for future exercises to work properly. 
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Create a Flat File Format 


Business Example 


In addition to the main databases for source information, records for orders are stored in flat 
files. You need to extract data from these flat files, and so must create the appropriate file 
format for the extraction. 


1. Create a file format Orders_Format for a flat file containing orders, so that you can use it 
as a source object for extraction. 


2. Adjust the datatypes for the columns proposed by the Designer based on their content. 


Table 2: Column Attributes Values 


Column Datatype Field Size 


COUNTRY 





3. Create a flat file format called Order Delivery Format. 


4. Adjust the datatypes for the columns proposed by the Designer based on their content. 
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Create a Flat File Format 


Business Example 


In addition to the main databases for source information, records for orders are stored in flat 
files. You need to extract data from these flat files, and so must create the appropriate file 
format for the extraction. 


1. Create a file format Orders_Format for a flat file containing orders, so that you can use it 
as a source object for extraction. 


a) In the Local Objects Library, choose Formats tab. 
b) Right click the Flat Files node and choose New. 
c) Enter Orders_Format as the format name. 


d) In the Data File(s) section, use the drop down menu to change Location to Job 
Server. 


e) In the Root Directory field, enter \\WDFLBMT5074\CourseFiles\DataServices 
\Activity Source. 


f) In the File name field, enter orders 12 21 06.txt. 


A pop-up message “Overwrite the current schema with the schema from the file you 
selected?” opens. Choose Yes. 


g) Inthe Delimiters section, in the Column field, use the drop down menu to change the 
file delimiter to Semicolon . 


A pop-up message “Overwrite the current schema with the schema from the file you 
selected?” opens. Choose Yes. 


h) In the Input/Output sections, in the Skip Row Header field, use the dropdown menu to 
change the value to Yes. 


i) Save your work. 


2. Adjust the datatypes for the columns proposed by the Designer based on their content. 


Table 2: Column Attributes Values 
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Column Datatype Field Size 





a) Inthe Column Attributes pane, change the field datatypes to the datatypes in the 
Column Attributes Values table. 


b) Inthe ORDERDATE field, to change the format of the date, enter dd-mon-yyyy . 
c) ChooseSave and close. 

d) Right click your new file format Orders Format and choose View Data. 

e) Open the Start menu and select the link Course Files-WDFLBMT5074. 


f) In the dialog box, enter the following credentials: 





3. Create a flat file format called Order Delivery Format. 


a) Inthe Local Object Library, choose the Formats tab. Right click Flat Files and choose 
New. 


b) Inthe File Format Editor, in the Type field, enter Delimited. 
c) Inthe Name field, enter Order Delivery Format. 
d) In the Data File(s) section, in the Location field, enter Job Server. 


e) Inthe Root Directory field, enter D: \CourseFiles\DataServices 
\Activity Source. 


f) In the File Name(s) field, enter Order Delivery 04 20 07.txt. 
g) Inthe Delimiters section, in the Column field, enter semicolon. 
h) When prompted to overwrite the schema, choose Yes. 

i) In the Input/Output section, in the Skip Row Header field, choose Yes. 
j) When prompted to overwrite the schema, choose Yes. 


4. Adjust the datatypes for the columns proposed by the Designer based on their content. 
a) Inthe Column Attributes pane, change these field datatypes: 


SHIPPERNAME varchar(50) 
SHIPPERADDRESS varchar(50) 
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SHIPPERFAX varchar(20) 
SHIPPERPOSTALCODE varchar(15) 





b) Choose Save — Close. 
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Create a Basic Data Flow 


Business Example 


You are an ETL developer working on a data warehousing project. You need to load data from 
various flat files to consolidate order and delivery data, so as to create a table for North 
American customers. 


A query transform will be used to join multiple files from file formats. 


Task 1: Create a Project 


1. Create anew project called Omega. 


Task 2: Create a Basic Data Flow 


1. 


7: 


In the Omega project, create a new batch job called Alpha_NACustomer_Job with anew 
data flow called Alpha_NACustomer_DF. 


. In the workspace for Alpha_NACustomer_DF, add the customer table from the Alpha 


datastore as the source object. 


. Create anew template table alpha_NA_customer in the Delta datastore as the target 


object. 


. Add the Query transform to the workspace between the source and target. 


. Inthe transform editor for the Query transform, map all columns from the Schema In to 


the Schema Out. 


. Use a WHERE clause to select only customers in North America (North American 


countries are United States, Canada, and Mexico which have COUNTRYID values of 1, 2, 
and 11). 


Save and execute the Alpha_NACustomer_Job. 


Task 3: Create a New Batch Job 


1. 


In the Omega project, create a new batch job called Alpha_Orders_Job with a new data 
flow called Alpha_Orders_DF. 


. Design the Alpha_Orders_DF data flow, with file formats, a query transform, and a target 


template table. 


. In the Delta datastore, create a new template table, called A11 Orders, as the target 


object. 


. Add the query transform to the workspace, and connect both sources toit. 


. Edit the source file formats in the data flow, so as to use all three related orders and order 


delivery flat files. 
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6. Edit the source file formats in the data flow to use all three related orders and order 
delivery flat files. 
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Create a Basic Data Flow 


Business Example 


You are an ETL developer working on a data warehousing project. You need to load data from 
various flat files to consolidate order and delivery data, so as to create a table for North 
American customers. 


A query transform will be used to join multiple files from file formats. 


Task 1: Create a Project 


1. Create anew project called Omega. 


a) Inthe Project menu, choose New — Project. 
b) In the New Project dialog box, in the Project Name field, enter Omega. 


c) Choose Create. 
The new project appears in the Project area. 


Task 2: Create a Basic Data Flow 


1. In the Omega project, create a new batch job called Alpha_NACustomer_Job with anew 
data flow called Alpha_NACustomer_DF. 


a) Inthe Project area, right click the project name. 
b) Choose New Batch Job. 


c) Name the job Alpha_NACustomer_Job and, on your keyboard, press Enter. 
The job should open automatically. If it does not, open it by double-clicking. 


d) In the tool palette, choose the Data Flow icon. 
e) Click the workspace where you want to add the data flow. 


f) Name the data flow Alpha_NACustomer_DF and press Enter. 
The job should open automatically. If it does not, open it by double-clicking. 


2. Inthe workspace for Alpha_NACustomer_DF, add the customer table from the Alpha 
datastore as the source object. 


a) Inthe Local Object Library , select the Datastores tab. 
b) Select the customer table from the Alpha datastore. 
c) Drag the table to the data flow workspace and choose Make Source. 


3. Create anew template table alpha_NA_customer in the Delta datastore as the target 
object. 


a) To add a new template table to the workspace, in the tool palette, choose the Template 
Table icon, and click the workspace. 
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b) In the Create Template dialog box, in the Table name field, enter 
alpha NA customer. 


c) In the /n datastore field, choose Delta from the dropdown list. 
d) Choose OK. 


4. Add the Query transform to the workspace between the source and target. 


a) To add anew Query template to the data flow, in the tool palette, choose the Query 
Transform icon, and click the workspace. 


b) To connect the source table to the Query transform, select the source table, hold down 
the left mouse button, drag the cursor to the Query transform, and release the mouse 
button. 


c) To connect the Query transform to the target template table, select the Query 
transform, hold down the left mouse button, drag the cursor to the target table, and 
release the mouse button. 


5. In the transform editor for the Query transform, map all columns from the Schema In to 
the Schema Out. 


a) To open the Query Editor , in the data flow workspace, double click the Query 
transform. 


b) To select all columns in the Schema In , choose the CUSTOMERID column, hold the 
shift key, and choose the PHONE column. 


c) Drag all columns to the Schema Out. 


6. Use a WHERE clause to select only customers in North America (North American 
countries are United States, Canada, and Mexico which have COUNTRYID values of 1, 2, 
and 11). 


a) Inthe Query editor, choose the WHERE tab. 
b) Enter the where clause, customer.COUNTRYID in (1,2,11). 


7. Save and execute the Alpha_NACustomer_Job. 
a) In the main menu, choose Project — Save All. 


b) To save all changes, choose OK. 

c) Inthe Project Area , right click the Alpha_NACustomer_Job and choose Execute. 
d) To accept the default execution properties, choose OK. 

e) To return to the Job workspace, in the tool bar, choose the Back icon. 

f) To return to the Data Flow workpace, double click the Data Flow. 


g) To view the template table data, select the small magnifying glass in the lower right- 
hand corner of the template table. 


h) Confirm that 22 records were loaded. 
i) Close the table. 
j) Close the Data flow. 
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Task 3: Create a New Batch Job 


1. In the Omega project, create a new batch job called Alpha_Orders_Job with a new data 
flow called Alpha_Orders_DF. 


a) Inthe Project area, right click the Omega project and choose New Batch Job. 


b) Enter the name Alpha_Orders_Job. 
The job should open automatically. If it does not, double click it. 


c) Inthe Tool Palette, choose the Data Flow button and click in the Job workspace. 
d) Enter the name Alpha_Orders_DF. 


2. Design the Alpha_Orders_DF data flow, with file formats, a query transform, and a target 
template table. 


a) Inthe Alpha_Orders_DF workspace, add the following file formats as source objects: 
e Orders_Format 
e Order_Delivery_Format 

b) Double click the Alpha_Orders_DF workspace to open it. 


c) Inthe Local Object Library, choose the Formats tab. Select the Orders_Format file 
format and drag it to the Data Flow workspace. 


d) Choose Make Source. 


e) On the Formats tab, select the Order_Delivery_Format file format and drag it to the 
Data Flow workspace. 


f) Choose Make Source. 


3. In the Delta datastore, create a new template table, called A11 Orders, as the target 
object. 


a) Inthe Too! Palette, choose the Template Table button. Click the workspace to add a 
new template table to the data flow. 


b) Inthe Create Template dialog box, enter the name All_Orders. 
c) Inthe Datastore field, enter Delta. 
d) Choose OK. 


4. Add the query transform to the workspace, and connect both sources to it. 


a) Inthe Too! Palette, choose the Query Transform button. Click in the Data Flow 
workspace to add a query. 


b) To connect the source file formats (Orders_Format and Order_Delivery_Format) to the 
query, select the sources, hold down the mouse button, drag the cursor to the query 
transform, and release the button. 


c) Double click the query to open the Query Editor. 
d) Inthe Query Editor, choose the FROM tab. 


e) Inthe Input Schema workspace, select the ORDER_DELIVERY_FORMAT.ORDERID field 
and drag it into the WHERE workspace. 
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f) Enter the equal sign =. 


g) To complete the expression, in the Input Schema workspace, select the 
ORDERS_FORMAT.ORDERID field and drag it into the FROM workspace. 


The expression should be as follows: Order Delivery Format.ORDERID = 
Orders Format .ORDERID. 


This will join the data in the formats on the OrderlD values. 


h) In the Query Editor, in the Input Schema workspace, choose the following fields and 
drag them to the Output Schema workspace: 


Orders_Format ORDERDATE 
Orders_Delivery_Format SHIPPERNAME 
Orders_Delivery_Format SHIPPERADDRESS 
Orders_Delivery_Format SHIPPERCITY 
Orders_Delivery_Format SHIPPERCOUNTRY 
Orders_Delivery_Format SHIPPERPHONE 
Orders_Delivery_Format SHIPPERFAX 
Orders_Delivery_Format SHIPPERREGION 
Orders_Delivery_Format SHIPPERPOSTALCODE 


This creates the necessary mapping. 











i) In the Output Schema workspace, right click the ORDERDATE field. Choose New 
Output Column — Insert Above. 


j) Specify the following data: 
Field Value 


Field Name ORDER_TAKEN_BY 


k) Choose OK. 





D In the Input Schema workspace, select Orders_Format.EMPLOYEEID and drag it to the 
ORDER_TAKEN_BY field in the Output Schema. 


m) In the Output Schema workspace, right click the ORDERDATE field. Choose New 
Output Column — Insert Above. 


n) Specify the following data: 
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Field Name ORDER_ASSIGNED _TO 


22 


o) Choose OK. 





p) Inthe /nput Schema, select the Orders_Format.EMPLOYEEID and drag it to the 
ORDER_ASSIGNED_TO field in the Output Schema. 


q) Close the editor. 


. Edit the source file formats in the data flow, so as to use all three related orders and order 


delivery flat files. 
a) Return to the Alpha_Orders_DF data flow workspace. 


b) Double click the Orders_Format source object to edit it. 


c) In the Data File(s) section, change the File Name(s) field to orders_*. txt. 


Note: 


The asterisk character acts as a wildcard. 





d) Inthe Error Handling section, change Capture Data Conversion Errors to Yes. 


Note: 


Do not change any other settings in the Error Handling section. 





e) Close the editor. 


. Edit the source file formats in the data flow to use all three related orders and order 


delivery flat files. 
a) Double-click the Orders_Delivery_Format source object to edit it. 


b) In the Data File(s) section, change the File name(s) field to Order _Delivery *.txt . 


Note: 


The asterisk character acts as a wildcard. 





c) Inthe Error Handling section, change Capture Data Conversion Errors to Yes. 


Note: 


Do not change any other settings in the Error Handling section. 





d) Close the editor. 
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Use the Interactive Debugger 


Business Example 


To ensure that your job is processing the data correctly, you wan to run the job in debug 
mode. To minimize the data you have to review in the interactive debugger, you set the debug 
option process to show only records from an individual CountrylD field value. 


Note: 


When the data values include ##, replace ## with the number that your instructor 
has provided to you. 





1. Inthe Cloud/WTS environment, the Designer will not allow multiple users to share the 
interactive debugger port. Change the interactive debugger port in Designer options. 


2. Execute the Alpha_NACustomer_Job in debug mode with a subset of records. In the 
workspace for the Alpha NACustomer Job, add a filter between the source and the 
Query transform to filter the records, so that only customers from the USA are included in 
the debug session. 


3. Once you have confirmed that the structure appears correct, you execute another debug 
session with all records, breaking after every row. 


Execute the Alpha_NACustomer_Job again in debug mode using a breakpoint to stop the 
debug process after a number of rows. 
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Use the Interactive Debugger 


Business Example 


To ensure that your job is processing the data correctly, you wan to run the job in debug 
mode. To minimize the data you have to review in the interactive debugger, you set the debug 
option process to show only records from an individual CountrylD field value. 


Note: 


When the data values include ##, replace ## with the number that your instructor 
has provided to you. 





1. In the Cloud/WTS environment, the Designer will not allow multiple users to share the 
interactive debugger port. Change the interactive debugger port in Designer options. 


a) From the main menu, choose Tools — Options. 
b) From the Designer options, choose Environment . 


c) Inthe Interactive Debugger field, enter port number 60## . 


A dialog box with the message “Overwrite job server option parameters (BODI 
1260099)” opens. To continue, choose Yes. 


d) To save changes, choose OK. 
2. Execute the Alpha_NACustomer_Job in debug mode with a subset of records. In the 
workspace for the Alpha_NACustomer_Job , add a filter between the source and the 


Query transform to filter the records, so that only customers from the USA are included in 
the debug session. 


a) Open the workspace for the Alpha_NACustomer_DF. 


b) Right click the connection between the source table and the Query Transform and 
choose Set Filter/Breakpoint . 


c) Inthe Filter window, select the Set checkbox 
d) Inthe Column field, from the drop-down list, choose customer. COUNTRYID. 
e) In the Operator field, from the drop-down list, choose = (Equals operator). 


f) In the Value field enter 1. 
This represents the country U.S.A. 


g) Choose OK. 


h) In the Project Area, right-click the Alpha_NACustomers_Job and choose Start 
debug. 
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i) In the Debug Properties dialog box, choose OK. 


Debug mode begins and all other Designer features are set to read-only. A Debug icon 
is visible in the task bar while the debug is in progress. 


You can specify many of the same properties as you can when executing a job without 
debugging. In addition, you can specify the number of rows to sample in the Data 
sample rate field. 


When the job is finished, a dialog box opens, with the question “Do you want to exit the 
Debugger now?” 


j) To stay in debug mode, choose No. 


k) Toclose the trace window and return to the job workspace, in the tool bar, choose 
Back. 


1) To open the data flow workspace, double click the data flow. 


m) Choose the magnifying glass between the Query transform and the target table. 
You should see that only five records are returned to the template table. 


n) Close the display. 
o) To exit debug mode, from the menu, choose Debug — Stop Debug. 


3. Once you have confirmed that the structure appears correct, you execute another debug 
session with all records, breaking after every row. 


Execute the Alpha_NACustomer_Job again in debug mode using a breakpoint to stop the 
debug process after a number of rows. 


a) In the workspace for the Alpha_NACustomer_DF , right click the connection between 
the source table and the Query transform, and choose Remove Filter 


b) Right click the connection between the source table and the Query transform, and 
choose Set Filter/Breakpoint . 


c) In the Breakpoint window, select the Set checkbox. 


d) To enable breaking the debug session during processing, select the checkbox Break 
after number of rows. 


e) In field to the right of Break after number of rows enter 20. 

f) Choose OK. 

g) Inthe Project Area , right-click the Alpha_NACustomer_Job and choose Start debug . 
h) In the Debug Properties dialog box, choose OK . 


i) Save your work. 
Debug mode begins, and stops after processing 20 rows. 


j) In the data view, select the All checkbox. 
You see 21 records. 


k) Deselect the All checkbox. 
You see only the 21st record. 


D To discard the record from the table, select it and choose Discard . 
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The record field values now appear as if a line has been drawn through each value. 


m) To continue processing, choose Debug — Continue. 
The next row is displayed 


n) Continue until you get a message that the job is finished. 
o) To exit debug mode, choose Debug — Stop Debug . 


p) Toremove the breakpoint from the data flow, right-click the connection, and choose 
Remove Breakpoint . 


q) Inthe data flow workspace, choose the magnifying glass between the Query transform 
and the target table to view the table records. Note that only 21 of 25 rows were 
returned, because you rejected one record. 


r) Close the display. 


s) Save your work. 
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Use Auditing in a Data Flow 


Business Example 


You must ensure that all records from the Customer table in the Alpha database are being 
moved to the Delta staging database using the audit logs. 


In the Local Object Library, replicate the Alpha_NACustomer_DF data flow. Name the 
replicated data flow Alpha_AuditCustomer DF. Add the replicated data flow to a new job, 
Alpha_AuditCustomer_Job. Set up auditing on the data flow Alpha_AuditCustomer_DF 


by adding an audit rule to compare the total number of records in the source and target 
tables. 


1. Replicate the Audit_NACustomer_DF data flow. 
2. Create a new batch job Alpha_AuditCustomer_Job. 
3. Add the Alpha AuditCustomer DF to the Alpha AuditCustomer Job. 


4. Add audit labels in the Alpha_AuditCustomer_DF data flow to count the total number of 
records in the source and target tables. 


5. Construct an audit rule that an exception must be entered into the log if the count from 
both tables is not the same. 


6. Enable auditing for the execution of the Alpha_AuditCustomer Job. 


7. Remove the audit feature from the dataflow. 
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Use Auditing in a Data Flow 


Business Example 


You must ensure that all records from the Customer table in the Alpha database are being 
moved to the Delta staging database using the audit logs. 


In the Local Object Library, replicate the Alpha_NACustomer_DF data flow. Name the 
replicated data flow Alpha_AuditCustomer DF. Add the replicated data flow to a new job, 
Alpha_AuditCustomer_Job. Set up auditing on the data flow Alpha_AuditCustomer_DF 
by adding an audit rule to compare the total number of records in the source and target 
tables. 
1. Replicate the Audit_NACustomer_DF data flow. 
a) Inthe Local Object Library Data Flow tab right click the Alpha_NACustomer_DF data 
flow and choose Replicate . 
b) Rename the copied data flow Alpha_AuditCustomer_DF. 


2. Create anew batch job Alpha_AuditCustomer_Job. 
a) Right click the Omega project in the Project Area. 


b) Choose New Batch Job 
c) Name the new job Alpha_AuditCustomer_Job. 


3. Add the Alpha AuditCustomer DFiotheAlpha AuditCustomer Job. 
a) Drag the Alpha AuditCustomer_DF from the Local Object Library to the 
Alpha AuditCustomer Job workspace. 
4. Add audit labels in the Alpha AuditCustomer_DF data flow to count the total number of 
records in the source and target tables. 
a) In the Local Object Library , choose the Data Flow tab. 


b) Right click the data flow Alpha AuditCustomer_DF and choose Audit . 


The Audit dialog box displays with a list of the objects that you can audit with any audit 
functions and labels for those objects. 


c) On the Label tab, right click the source table, customer , and choose Count . 


d) On the Label tab, right click the target table, Alpha NA customer, and choose 
Count . 


5. Construct an audit rule that an exception must be entered into the log if the count from 
both tables is not the same. 


a) Inthe Rule tab, under Auditing Rules , choose Add. 
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The expression editor opens. It contains three drop-down lists where you specify the 
audit labels for the objects that you want to audit and choose the expression to use 
between these labels. 


b) In the left drop down list, choose the audit label $Count_customer for the source table. 
c) In the operator drop down list, choose the operator equal (=). 


d) In the right drop down list, choose the audit label $count_Alpha_NA_customer for the 
target table. 


e) Under Action on failure, select the Raise exception checkbox. 
f) Choose Close . 


6. Enable auditing for the execution of the Alpha AuditCustomer Job. 
a) Right-click the Alpha_AuditCustomer_Job. 


b) Choose Execute . 


c) In the Execution Properties dialog box, choose the Execution Options tab, and select 
the Enable auditing checkbox. 


d) Choose the Trace tab and choose Trace Audit Data . 
e) In the Value field, use the drop down to change the value from No to Yes. 


f) Choose OK. 
You see the audit rule fail. 


7. Remove the audit feature from the dataflow. 
a) Inthe Local Object Library ,choose the Data Flow tab. 


b) Right click the data flow Alpha_AuditCustomer_DF and choose Audit . 


c) In the Rule tab, under Auditing Rules , choose the auditing rule that you created, and 
choose Delete . 


d) In the Label tab, right click the source table Customer and choose Count. 
This action toggles the label off. 


e) In the Label tab, right click the source table Alpha NA customers and choose Count. 


f) In the Label tab , right click the source table Alpha_Other_customer and choose 
Count . 


g) Close and save your work. 
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Use the lookup_ext() Function 


Business Example 


Inthe Alpha Acquisitions database, the country for a customer is stored in a separate table 
and referenced with a code. To speed up access to information in the data warehouse, this 
lookup should be eliminated. 


Use the lookup_ext function to exchange the ID for the region and country names in the 
customers table for Alpha with the actual value from the region and country tables. 


1. Inthe Alpha_NACustomer_DF workspace, delete an existing expression for the Country 
column in the Query transform. 


2. Use the Functions wizard to create a new lookup expression using the /ookup_ext function. 


3. In the Alpha_NACustomer_DF workspace, add a new function call to add COUNTRYNAME 
below the COUNTRYID field. 


4. Execute the Alpha_NACustomer_Job with the default execution properties after saving 
all objects you have created. 
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Use the lookup_ext() Function 


Business Example 


In the Alpha Acquisitions database, the country for a customer is stored in a separate table 
and referenced with a code. To speed up access to information in the data warehouse, this 
lookup should be eliminated. 


Use the lookup_ext function to exchange the ID for the region and country names in the 
customers table for Alpha with the actual value from the region and country tables. 


1. Inthe Alpha_NACustomer_DF workspace, delete an existing expression for the Country 
column in the Query transform. 


a) Inthe Alpha_NACustomer_DF workspace, to open the transform editor, double click 
the Query transform. 


b) In the Query Editor, in the output schema, choose the field REGIONID. 
c) In the Mapping tab for the REGIONTID field, delete the existing expression. 
d) Change output column REGIONID to REGION of type varchar with a length of 50. 


2. Use the Functions wizard to create a new lookup expression using the lookup_ext function. 
a) In the Mapping tab, choose Functions. 


b) In the Select Function dialog box, choose Lookup Functions. 


c) Choose the lookup_ext function and choose Next. 


d) In the Lookup_ext - Select Parameters dialog box, enter the following parameters: 


fonts PE 


seo ES 
CE o 


e) To close the editor, choose Finish. 





3. Inthe Alpha_NACustomer_DF workspace, add a new function call to add COUNTRYNAME 
below the COUNTRYID field. 
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a) Inthe Alpha_NACustomer_DF workspace, double click the Query transform to open 
the transform editor. 


b) In the Query Editor, in the output schema, choose the field COUNTRYID. 


c) Right click COUNTRYID to select the option, New Function Call, from the context 
menu. 


d) Inresponse to the popup box, select Insert Below. 


e) In the dialog box, Select Function, under the column, Function categories:, select the 
category, Lookup functions. 


f) Select the Lookup Function in the column Function Name: and click Next. 


g) Inthe Lookup_ext - Select Parameters dialog box, enter the following parameters from 
the table: 


fenitn PS 


eo EE 
CS PE 


h) To close the editor, choose Finish. Observe how Data Services places the result of the 
lookup function under the original field. In this way, you can retain the original field and 
enhance the output with the name field. 





4. Execute the Alpha NACustomer_ Job with the default execution properties after saving 
all objects you have created. 


a) Right click the Alpha_NACustomer_Job in the Omega project and choose Execute . 


Data Services prompts you to save any objects that have not been saved. Choose OK. 
b) Touse the default execution properties, choose OK. 
c) Toreturn to the job workspace, on the toolbar, choose the Back icon. 
d) To open the data flow workspace, double click the data flow. 


e) Right click the target table and choose View Data. 


Note that the country codes are replaced by the country names, and the region codes 
are replaced by the region names. 


f) Close the display. 
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Use Aggregate Functions 


Business Example 


You must calculate the total value of all orders, including their discounts, for reporting 
purposes. Currently these details are found in different tables. 


Use the sum function to calculate the total value of orders in the Order_Details table. 


1. Create a new batch job called Alpha_Order_Sum_Job with a data flow 
Alpha Order Sum DF. 


2. Inthe transform editor for the Query transform, propose a join between the two source 
tables. 


3. Inthe Query transform, create a new output column TOTAL VALUE, which will hold the 
new calculation. 


4. Map the TOTAL VALUE column using the sum function. The value is the product of the 
quantity from the order_details table and the cost from the products table, multiplied 
by the discount from the order_details table. 


5. Now that the expression can calculate the total of the order values, make it possible for 
the Query to begin at the first order through the end of the records in the table by using 
the Group By tab. 


6. Execute the Alpha Order Sum Job with the default execution properties after saving all 
of the objects that you have created. 
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Use Aggregate Functions 


Business Example 


You must calculate the total value of all orders, including their discounts, for reporting 
purposes. Currently these details are found in different tables. 


Use the sum function to calculate the total value of orders in the Order Details table. 


1. Create a new batch job called Alpha Order Sum Job with a data flow 
Alpha Order Sum DF. 


a) In the Project area , right click your Omega project and choose New batch job. 


b) Enter the job name Alpha Order Sum Job. 


c) Inthe Alpha Order Sum Job workspace, from the toolbar, choose the Data Flow 
icon. 


d) To add the data flow to your new job, click in the workspace, and enter the name 
Alpha Order Sum_DF. 


e) Inthe Local Object Library , choose the Datastores tab. 


f) From the Alpha datastore, select the Order Details table, drag it to the 
Alpha Order Sum DF workspace and choose Make Source . 


g) From the Alpha datastore, select the product table, drag it to the 
Alpha Order Sum DF workspace and choose Make Source . 


h) From the tool palette, choose the Template Table icon. 
i) To place the template table, click in the Alpha_Order_Sum_DF workspace. 


j) Inthe Create Template dialog box, in the Table name field, enter order_sum. and 
change the In datastore field to Delta. 


k) From the tool palette, select the Query Transform icon. 
I) To place the Query Transform, click in the Alpha_Order_Sum_DF workspace. 


m) To connect the Order Details table to the Query Transform , select the source 


table, hold down the mouse button, drag it to the Query Transform , and release the 
mouse button. 


n) Toconnect the Product table to the Query Transform , select the source table, hold 


down the mouse button, drag it to the Query Transform , and release the mouse 
button. 


o) To connect the Query Transform, select the Query Transform , hold down the mouse 
button, drag it to the order_sum table, and release the mouse button. 


34 
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2. Inthe transform editor for the Query transform, propose a join between the two source 
tables. 


a) To open the Query Editor double click the Query . 
b) Choose the From tab. 


c) In Join Pairs section of the Where tab, for the left source, choose the Order_Details 
table and for the right source, choose the Product table. 
The Designer should enter the following code: product. PRODUCTID = 
order details.PRODUCTID. 
If the Designer does not respond, type in the expression exactly. 


3. Inthe Query transform, create a new output column TOTAL VALUE, which will hold the 
new calculation. 
a) To map the ORDERID column from the input schema to the same field in the output 
schema, select ORDERID and drag to the output schema 
b) In the output schema, right click ORDERID and choose New output column. 
c) Choose Insert Below . 
d) Enter the name TOTAL_VALUE with a data type of double. 
e) Choose OK. 


4. Map the TOTAL VALUE column using the sum function. The value is the product of the 
quantity from the order_details table and the cost from the products table, multiplied 
by the discount from the order_details table. 

a) On the Mapping tab of the TOTAL VALUE column, enter the expression: 
sum ( (order details .QUANTITY*product.COST) *order details .DISCOUNT) 


Note: 

The expression could also look like this: 

sum(order_details. QUANTITY*product.COST*order_details. DISCOUNT). 
With multiplication, it does not matter the order in which the operands are 
placed. 


Note: 
If you validate the expression, the validation will fail. Once you complete the 
next step, the validation will pass. 





5. Now that the expression can calculate the total of the order values, make it possible for 
the Query to begin at the first order through the end of the records in the table by using 
the Group By tab. 


a) Inthe Query Editor , select the Group By tab. 


b) In the Schema In column, select the ORDERID field from the ORDER DETAILS table and 
drag it to the Group By tab. 


c) Close the Editor. 
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6. Execute the Alpha_Order_Sum_Job with the default execution properties after saving all 
of the objects that you have created. 


a) Inthe Omega project, right click the Alpha Order Sum Job. 


b) Choose Execute . 


Data Services prompts you to save any objects that have not been saved. Choose OK. 
c) Touse the default execution properties, choose OK . 
d) Return to the job workspace, 
e) Open the data flow workspace. 
f) Right click the target table and choose View data . 
g) Confirm that order 11146 has 204000 as a total value. 
h) Close the display. 
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Create a Custom Function 


Business Example 


The Marketing department would like to send special offers to customers who have placed a 
specified number of orders. This can be done by creating a custom function that must be 
called when a customer order is placed. You want to create a custom function to accept the 
input parameters of the Customer ID, and the number of orders required to receive a special 
order, check the Orders table, and then create an initial list of eligible customers. 


1. Inthe Local Object Library, create a new customer function called CF_MarketingOffer. 


2. Create a new batch job and data flow, called Alpha Marketing Offer Job and 
Alpha Marketing Offer DF respectively, and anew global variable $G Num to Qual. 


3. In the job workspace, define a script to define the global variable and attach the script to 
the data flow. 


4. Define the data flow with the customer table from the Alpha datastore as a source, a 
template table as a target and two Query transforms between the source and target. 


5. Execute Alpha_Marketing_Offer_Job with the default properties and view the results. 
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Create a Custom Function 


Business Example 


The Marketing department would like to send special offers to customers who have placed a 
specified number of orders. This can be done by creating a custom function that must be 
called when a customer order is placed. You want to create a custom function to accept the 
input parameters of the Customer ID, and the number of orders required to receive a special 
order, check the Orders table, and then create an initial list of eligible customers. 


1. In the Local Object Library, create a new customer function called CF_MarketingOffer. 


a) In the Local Object Library, choose the Custom Functions tab. 

b) Enter the nameCF MarketingOffer, and choose Next. 

c) Inthe Smart Editor, choose the Variables tab. 

d) Right click Parameters and choose Insert. 

e) In the Parameter Properties dialog box, enter the name $P_CustomerID. 
f) In the Data type field, enter int 

g) Inthe Parameter type field, enter Input. 

h) Choose OK. 
i) Right click Parameters and choose Insert. 
j) Inthe Parameter Properties dialog box, enter the name $P_Orders. 
k) In the Data type field, enter int. 
D Inthe Parameter type field, enter Input. 

m) Choose OK. 


n) Inthe workspace of the Smart Editor, enter the following code on three separate lines: 
If (SQL ('Alpha' , 'SELECT COUNT(*) From orders WHERE CUSTOMERID 
= [$P_CustomerID]') >= $P_Orders) 

Return 1; 


Else return 0; 


Note: 


Do not use the ifthenelse function. Type inthe if function. 





This code defines the custom function as a conditional clause. The conditional clause 
specifies that, if the number of rows in the orders table is equal to the value of the 
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parameter $P_Orders for the Customer ID, the function should return a 1. Otherwise, 
it should return 0. 


o) Choose Validate, and make any necessary corrections. 


Note: 
If your function contains syntax errors, Data Services displays a list of 


those errors in an embedded pane below the editor. To see where the error 
occurs in the text, double-click an error. The Smart Editor redraws to show 
you the location of the error. 





p) Choose OK. 


2. Create a new batch job and data flow, called Alpha Marketing Offer Job and 
Alpha Marketing Offer DF respectively, and anew global variable $G Num to Qual. 


a) In the project area, right click the Omega project and choose New batch job 
b) Enter the name Alpha_ Marketing Offer Job. 

c) In the tool palette, select the Data Flow icon and click in the workspace. 
d) Enter the name Alpha Marketing Offer DF. 

e) Select the job Alpha Marketing Offer Job and choose Tools — Variables. 
f) Right click Global Variables and choose Insert. 
g) Right click the new variable and choose Properties. 

h) In the Global Variable Properties box, enter the name $G_ Num to Qual. 

i) In the Data type field, enter int. 

j) Choose OK. 

k) Close the display. 


3. In the job workspace, define a script to define the global variable and attach the script to 
the data flow. 


a) Inthe project area, choose the Alpha Marketing Offer Job. 

b) From the tool palette, choose the Script icon. 

c) To place the script, click in the workspace to the left of the data flow. 
d) Name the script CheckOrders. 

e) To open the script, double-click it. 


f) Enter the expression $G Num_ to Qual = 5; 
This creates an expression that defines the global variable as five orders to qualify for 
the special marketing campaign. 

g) Add a print statement to print the global variable to the log, as follows: 


print(the global variable value is assigned at the job level. the value is: '|| 
$G_Num_to_Qual); 
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h) Close the script and return to the job workspace. 


i) To connect the script to the data flow, select it and, while holding the mouse button, 
drag it to the data flow. Release the button to create the connection. 


4. Define the data flow with the customer table from the Alpha datastore as a Source, a 
template table as a target and two Query transforms between the source and target. 


a) From the Local Object Library, choose the Datastores tab. 


b) Inthe Alpha Datastore, select the customer table, drag it into the data flow 
workspace, and choose Make Source . 


c) Inthe tool palette, select the Template Table icon, and click in the workspace. 
This adds the template table to your data flow. 


d) Name the table offer mailing list, choose the Delta datastore, and choose OK. 


e) From the tool palette, select the Query Transform icon, and click in the data flow 
workspace. 


f) From the tool palette, again select the Query Transform icon and click in the data flow 
workspace. 


g) Toconnect the source table to the first query, select the table, and, while holding down 
the mouse button, drag it to the query. Release the button to create the connection. 


h) Toconnect the first query to the second query, select the first query and, while holding 
the mouse button, drag it to the second query. Release the button to create the 
connection. 


i) To connect the target table to the second query, select the second query and, while 
holding the mouse button, drag it to the target table. Release the button to create the 
connection. 


j) Open the Query Editor for the first query, and select the following input columns from 
the Schema In and drag them to the Schema Out on the Query node: 


+ CONTACTNAME 
« ADDRESS 
e CITY 
+ POSTALCODE 
k) Right click POSTALCODE, choose New Output Column, and choose Insert Below. 
1) Enter the column name OFFER_STATUS. 
m) In the Datatype field, enter int and choose OK. 
n) On the Mapping tab of the OFFER_STATUS column, choose Functions. 


o) Inthe Select Function, choose category Custom Functions, your custom function 
CF_MarketingOffer, and choose Next. 


p) In the Define Input Parameter(s) dialog box, in the $P_CustomerID field, choose the 
CUSTOMER table and then choose OK. 
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q) From the list of table fields, select CUSTOMERID and choose OK. 


You will be returned to the Function Wizard. 
r) Inthe $P_Orders field, choose the Smart Editor icon (the button with the three dots). 


s) On the Variables tab, expand the node for Global Variables, and then the node for your 
job. 


t) Right click the global variable $G Num to_Qual and choose Enter. 


u) To return to the Function Wizard, choose OK. 
The expression should look like this: 
CF MarketingOffer (CUSTOMERID, $G Num to Qual) 


v) Close the Query transform. 


w) Open the second Query and in the Query Editor, select the following input columns 
from the Schema In and drag them to the Schema Out: 


+ CONTACTNAME 
e ADDRESS 

e CITY 

+ POSTALCODE 


x) In the WHERE tab, enter an expression to select only those records where 
OFFER_STATUS has a value of one. 


y) From the Schema In, select the input column OFFER_STATUS, drag it into the WHERE 
tab workspace, and enter =1. 
The expression should be: Query.OFFER_STATUS = 1. 
This will select only those records where OFFER_STATUS has a value of one. 


5. Execute Alpha_Marketing_Offer_Job with the default properties and view the results. 
a) In the project area, select Alpha_Marketing_Offer_Job and choose Execute. 


If you have unsaved changes, a Save All Changes and Execute dialog box opens. To 
continue, choose, Yes. 


b) To accept the default execution properties, choose OK. 


c) Return to the job workspace. 


Note: 


Note the Print Function statement in the log. 





d) Open the data flow workspace. 


e) Right click the target table to choose View Data. 
You should have one output record for contact Lev M. Melton in Quebec. 


f) Close the display. 
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Business Example 


Users of employee reports have requested that employee records in the data mart contain 
only records for current employees. You use the Map Operation transform to change the 
behavior of loading so the resulting target conforms to this business requirement by removing 
any employee records that contain a value in the discharge data column of the source data. 


1. Create a new batch job Alpha Employees Current Job with a data flow 
Alpha Employees Current _DF, which contains a Map Operation transform. 


2. Add the Map Operation transform to the data flow, change the output operation code of 
NORMAL to DELETE, Save all objects and execute the job. 


3. Save all objects and execute the Alpha_Employees Current Job. 
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Use the Map Operation Transform 


Business Example 


Users of employee reports have requested that employee records in the data mart contain 
only records for current employees. You use the Map Operation transform to change the 
behavior of loading so the resulting target conforms to this business requirement by removing 
any employee records that contain a value in the discharge data column of the source data. 


1. Create a new batch job Alpha_Employees_Current_Job with a data flow 
Alpha Employees Current _DF, which contains a Map Operation transform. 


a) In the project area right click the Omega project, choose New Batch job, and change the 
name to Alpha Employees Current Job. 


b) In the workspace for the job, from the tool palette, select the Data Flow icon and click 
in the workspace. Enter the name Alpha Employees Current DF. 


c) Open the data flow workspace and, from the Alpha datastore in the Local Object 


Library , select the Employee table, drag it into the workspace, and choose Make 
Source. 


d) From the HR_datamart datastore, select the EMPLOYEE table, drag it into the 
workspace, and choose Make Target. 


e) From the tool palette, choose the Query Transform icon and click in the workspace. 
f) Connect the source table to the Query transform. 
g) To open the Query Editor , double-click the Query. 


h) To only map the EMPLOYEEID column from the input schema to the output schema, in 
the Schema In, select EMPLOYEEID and drag it to the Schema Out. 


i) Select the WHERE tab. 


j) From the Schema In pane, drag the DISCHARGE_DATE column into the WHERE tab 

workspace. 

k) Complete the expression by entering is not null. 
The entire expression should be: 
employee.discharge date is not null 
This will select only those rows where the discharge date field is not empty. 

2. Add the Map Operation transform to the data flow, change the output operation code of 
NORMAL to DELETE, save all objects and execute the job. 
a) In the Local Object Library , select the Transform tab, and open the node Platform . 


b) Choose the Map Operation transform and drag it into the data flow workspace. 
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c) Connect the Query transform to the Map Operation transform, and connect the Map 
Operation transform to the target table. 


d) Open the Map Operation Transform Editor and, in the Map Operation tab, change the 
settings so that rows with /nput row type Normal have an Output row type Delete. 


3. Save all objects and execute the Alpha Employees Current Job. 


a) Inthe project area, right click Alpha Employees Current _Job and choose Execute . 


A Save all changes and execute dialog box opens. To continue, choose Yes . 
b) To use the default settings, in the Execution Properties dialog box, choose OK. 
c) Return to the job workspace. 


d) Inthe data flow workspace, choose the magnifying glass button on the source table. 
A large View Data pane appears beneath the current workspace area. 


e) Select the magnifying glass button on the target table. 


Two rows were filtered from the target table. Both of these records have 
discharge_date field entries. 


f) Close both displays. 


44 © Copyright. All rights reserved. 





Use the Validation Transform 


Business Example 


Order data is stored in multiple formats with different structures and different information. 
You need to learn how to use the Validation transform to validate order data from flat file 
sources and the Alpha Orders table before merging it. 


Task 1: Create a New Batch Job 
Create a new batch job called Alpha Orders Validated Job and two data flows, one 
named Alpha Orders Files DF and Alpha Orders _DB DF inthe Omega project. 


1. Create a new batch job Alpha_Orders_Validated_Job with a new data flow called 
Alpha Orders Files DF in the Omega project. 


2. Create a new data flow called Alpha_Orders DB DF inthe 
Alpha Orders Valiated Job workspace, 


Task 2: Design the Alpha_Orders_Files_DF Data Flow 
Design the data flow Alpha Orders Files DF with a table, a Query transform, a Validation 
transform and target template tables. 


1. Inthe workspace for Alpha Orders Files DF,addtheAll Orders template table, 
from the Delta datastore, as a source object. 


2. Create a new template table Orders Files Workin the Delta datastore as the target 
object. 


3. Create a new template table Orders Files No Fax inthe Delta datastore as the 
target object. 


4. Create new template table Orders Rule Violationin the Delta datastore as the 
target object. 


5. Add the Query transform to the workspace and connect both sources toit. 


6. Add the Validation transform to the workspace to the right of the Query transform and 
connect them. 


7. Add a validation rule to replace any NULL values with: No Current Empl. 


8. Add a validation rule for the shipper's fax to replace any NULL values with No Fax 
Number. 


9. Complete the data flow Alpha Orders Files DFbyconnecting the pass, fail, and rule 
violation outputs from the Validation Transform to the target template tables. 


Task 3: Design the Alpha_Orders_Files_DB_DF Data Flow 
Design the data flow Alpha_Orders_DB DF with the Orders table from the Alpha datastore, 
a Query Transform, a Validation Transform and target template tables. 
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10. 


. Inthe workspace for Alpha_Orders_DB DF, add the Orders table from the Alpha 


datastore, as a source object. 


. Create a new template table Orders_DB Work inthe Alpha_Orders DB DF workspace 


as a target object. 


. Create a new template table Orders DB No Faxinthe Delta datastore as a target 


object. 


. Create a new template table Orders DB Rule Violationinthe Delta datastore as the 


target object. 


. Add the Query transform to the workspace and connect it to the source table. 
. Add the Validation transform to the workspace to the right of the query and connect them. 
. Add a validation rule to replace any NULL values withNo Current Empl. 


. Add a validation rule for the shipper's fax to replace any NULL values withNo Fax 


Number. 


. Complete the data flow Alpha_Orders_DB_DF by connecting the pass, fail, and rule 


violation outputs from the Validation transform to the target template tables. 


Execute the Alpha_Orders_Validated_Job and view the differences between passing 
and failing records. 
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Use the Validation Transform 


Business Example 


Order data is stored in multiple formats with different structures and different information. 
You need to learn how to use the Validation transform to validate order data from flat file 
sources and the Alpha Orders table before merging it. 


Task 1: Create a New Batch Job 
Create a new batch job called Alpha Orders Validated Job and two data flows, one 


named Alpha Orders Files DFandAlpha Orders DB_DFin the Omega project. 
1. Create a new batch job Alpha Orders Validated Job with a new data flow called 
Alpha Orders Files DFinthe Omega project. 
a) In the Project area, right click the Omega project name and choose New Batch job . 


b) Enter the name Alpha_Orders_ Validated Job. 
The job should open automatically. If it does not, double-click it. 


c) In the tool palette, choose the Data Flow icon and click in the job workspace 
d) Enter the name Alpha_Orders Files DF. 


2. Create a new data flow called Alpha Orders DB DF inthe 
Alpha Orders Valiated Job workspace, 
a) In the tool palette, choose the Data Flow icon and click in the job workspace where you 
want to add the data flow. 


b) Enter the name Alpha_Orders_DB_DF and, on your keyboard, press the Enter key. 


Task 2: Design the Alpha_Orders_Files_DF Data Flow 
Design the data flow Alpha Orders Files DF witha table, a Query transform, a Validation 
transform and target template tables. 


1. Inthe workspace for Alpha Orders Files DF, add the All_Orders template table, 
from the Delta datastore, as a source object. 


a) To open the Alpha_Orders Files DF workspace double-click it. 


b) In the Local Object Library, choose the Datastore tab, select the Al! orders template 
table, and drag it to the data flow workspace. 


2. Create a new template table Orders Files Work in the Delta datastore as the target 
object. 


a) To add anew template table to the data flow, in the tool palette, choose the Template 
Table icon and click the workspace. 


b) In the Create Template dialog box, enter the name Orders Files Work. 


c) Inthe /n datastore field, enter Delta. 
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d) Choose OK. 


3. Create anew template table Orders Files No Fax inthe Delta datastore as the 
target object. 


a) To add a new template table to the data flow, in the tool palette, choose the Template 
Table icon and click the workspace. 


b) Inthe Create Template dialog box, enter the name Orders Files No Fax. 
c) Inthe In datastore field, enter Delta. 
d) Choose OK. 


4. Create new template table Orders Rule Violationinthe Delta datastore as the 
target object. 


a) To add a new template table to the dataflow, in the tool palette, choose the Template 
Table icon and click the workspace. 


b) Inthe Create Template dialog box, enter the name 
Orders Files Rule Violation. 


c) Inthe /n datastore field, enter Delta. 
d) Choose OK. 


5. Add the Query transform to the workspace and connect both sources to it. 
a) To add a query to the workspace, in the tool palette, choose the Query Transform icon 
and click in the data flow workspace. 


b) To connect the source template table, All_Orders, to the Query, select the source, hold 
down the mouse button, drag the cursor to the Query transform, and release the 
mouse button. 


c) To open the Query Editor , double-click the Query . 


d) Inthe Schema Out workspace, right-click the field ORDERDATE , choose New Output 
Column, and choose Insert Above . 


e) Enter the field name ORDER TAKEN BY, with a datatype of varchar and a length of 
15 and choose OK. 


f) To map ORDER TAKEN BY to Orders_Format.EMPLOYEEDD , in the input schema, 
select Orders_ Format .EMPLOYEEID and drag it to the ORDER TAKEN BY field in the 
output schema. 


g) Inthe Schema Out workspace, right-click the field ORDERDATE , choose New Output 
Column , and choose Insert Above 


h) Enter the field name ORDER_ASSIGNED_TO, with a datatype of varchar and a length 
of 15, and choose OK. 


i) To map ORDER ASSIGNED TOtOOrders Format.EMPLOYEEID , in the input 
schema, select Orders_Format .EMPLOYEEID and drag it to the 
ORDER ASSIGNED_TO field in the output schema. 


j) Close the editor. 
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6. Add the Validation transform to the workspace to the right of the Query transform and 
connect them. 


a) In the Local Object Library , choose the Transforms tab. 


b) In the Platform node , select Validation and drag it to the right of the Query in the data 
flow workspace. 


c) To connect the Query transform to the Validation , choose the Query, and hold down 


the mouse button while dragging the cursor to the Validation . Release the mouse 
button. 


7. Add a validation rule to replace any NULL values with: No Current Empl. 
a) To open the Validation Editor, double-click Validation. 


b) In the input schema, select the field ORDER_ASSIGNED_TO, and in the Validation Rules 
area, choose Add. 


c) Enter the name Orders Assigned To. 

d) Select the Enabled checkbox, if it not already selected. 

e) Select the Column Validation radio button. 

f) In the Column: field choose Query.ORDERS ASSIGNED TO. 


g) Inthe Condition: field, choose Exists in table. 
The Rules Editor opens. 


h) Inthe field, select the HR_datamart datastore and double-click to see its tables. 


i) Double-click the table EMPLOYEE to See its fields, choose the EMPLOYEEID field and 
choose OK. 


The resulting expression should be HR DATAMART . DBO . EMPLOYEE .EMPLOYEEID. 


j) In the Action on Fail field, set the action Send to Both. 
This sends to both Pass and Fail tables. 


k) To close the Rule Editor , choose OK. 
D In the If any rule fails and Send to Pass, substitute with: section, select Enabled . 


m) In the Column field, use the drop-down list to choose the field 
QUERY .ORDERS_ASSIGNED TO. 


n) In the Expression field , choose the ellipsis (...) icon, and, in the Smart Editor, enter the 
expression 'NO CURRENT EMPL' and choose OK. 


Note: 





You must use the single quotation marks before and after the string. 


8. Add a validation rule for the shipper's fax to replace any NULL values withNo Fax 
Number. 


a) In the input schema area, select the field SHIPPERFAX and, in the Validation Rules 
area, choose Add. 
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b) Enter the name Shipper Fax. 

c) Select the Enabled checkbox if it not already selected. 

d) To open rule editor, select the Column Validation radio button. 
e) Inthe Column: field choose Query . SHIPPERFAX . 

f) In the Condition: field, choose IS NOT NULL. 

g) Inthe field Action on Fail , set the action Send to Both. 


h) In the If any rule fails and Send to Pass, substitute with: section, select the check box 
Enabled . 


i) Inthe Column field, use the drop-down list to choose the field QUERY.SHIPPERFAX . 


j) Inthe Expression field , choose the ellipsis (...) icon. In the Smart Editor, enter the 
expression 'No Fax Number' and choose OK. 


Note: 





You must use the single quotation marks before and after the string. 


k) Choose OK and close the editor. 


9. Complete the data flow Alpha_Orders_Files_DF by connecting the pass, fail, and rule 
violation outputs from the Validation Transformto the target template tables. 


a) Return to the data flow workspace. 


b) Select the Validation Transform and drag it to the target template table 
Orders Files Work. 


c) Release the mouse and choose Pass. 


d) Select Validation Transform and drag it to the target template table 
Orders Files No Fax. 


e) Release the mouse and choose Fail . 


f) Select Validation Transform and drag it to the target template table 
Orders Files Rule Violation. 


g) Release the mouse and choose Rule Violation . 


Task 3: Design the Alpha_Orders_Files_DB_DF Data Flow 
Design the data flow Alpha_Orders_DB DF With the Orders table from the Alpha datastore, 
a Query Transform, a Validation Transform and target template tables. 


1. In the workspace for Alpha_Orders_DB DF, add the Orders table from the Alpha 
datastore, as a source object. 


a) Inthe Local Object Library , choose the Datastores tab. 


b) Inthe Alpha datastore, select the Orders table, drag it to the data flow workspace, 
and choose Make Source. 


2. Create anew template table Orders DB Work in the Alpha Orders DB DF workspace 
as a target object. 
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a) To add anew template table to the data flow, in the tool palette, choose the Template 
Table icon and click in the workspace. 


b) In the Create Template dialog box, enter the name Orders DB Work. 
c) Inthe In datastore field, choose the Delta datastore as the template table destination. 


d) Choose OK. 


3. Create a new template table Orders DB No_Fax in the Delta datastore as a target 
object. 


a) To add anew template table to the data flow, in the tool palette, choose the Template 
Table icon and click in the workspace. 


b) In the Create Template dialog box, enter the name Orders DB No Fax. 
c) In the /n datastore field, choose the Delta datastore as the template table destination. 
d) Choose OK. 


4. Create a new template table Orders DB Rule Violation in the Delta datastore as the 
target object. 


a) To add anew template table to the dataflow, in the tool palette, choose the Template 
Table icon and click in the workspace. 


b) In the Create Template dialog box, enter the name Orders DB Rule Violation. 
c) In the /n datastore field, enter Delta. 
d) Choose OK. 


5. Add the Query transform to the workspace and connect it to the source table. 


a) To add a Query transform to the data flow, in the tool palette, select the Query 
Transform icon and click in the workspace. 


b) To connect the source table to the query, select the table, and, holding down the 
mouse button, drag the cursor to the query. Then release the mouse button. 


c) To open the Query Editor , double-click the query. 


d) In the Query transform, to map all of the columns, except for EMPLOYEEID , from the 
input schema to the output schema, select the input schema field and drag it to the 
corresponding output schema field 


e) In the output schema, right-click the field ORDERDATE , choose New Output Column, 
and choose Insert Above . 


f) Name the new field ORDER_TAKEN_BY and choose datatype varchar and length15. 


g) Select Orders .EMPLOYEEDD in the input schema and drag it to the ORDER TAKEN BY 
field in the output schema. 
This maps the new ORDERS_TAKEN BY field to the orders .EMPLOYEEID field. 


h) In the output schema, right-click the field ORDERDATE and choose New Output Column, 
and choose Insert Above. 


i) Name the new field ORDER ASSIGNED _TO and choose datatype varchar and length 
15. 
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j) Select Orders .EMPLOYEEID in the input schema and drag it to the 
ORDER ASSIGNED_TO field in the output schema. 


This maps the new ORDERS_ASSIGNED_TO field to the Orders_Format .EMPLOYEEID 
field. 


k) Close the editor. 


. Add the Validation transform to the workspace to the right of the query and connect them. 


a) Inthe Local Object Library, choose the Transforms tab. 


b) Drag the Validation transform from the Platform node to the data flow workspace to 
the right of the query. 


c) To connect the query to the Validation transform, select the query, hold down the 
mouse button, drag the cursor to the Validation transform, and release the mouse 
button. 


. Add a validation rule to replace any NULL values withNo Current Empl. 


a) Double click the Validation to open the Transform Editor. 
b) Inthe input schema, choose the ORDER_ASSIGNED_TO field. 
c) Inthe Validation Rules area, choose Add. Enter the name: Orders Assigned To. 


d) Inthe Rules area, select the Enabled checkbox, if it is not already selected. 
Select the Column Validation radio button to open the Rule Editor. 


In the Column field, choose Query .ORDERS_ASSIGNED_TO. In the Condition field, 
choose Exists in table. 


e) Inthe next field, choose the HR_datamart datastore. Double click it to see the tables. 


f) Double click the EMPLOYEE table. Choose the EMPLOYEEID field and choose OK. 
You see the expression: HR_Datamart . dbo . EMPLOYEE . EMPLOYEEID 


g) Inthe Action on Fail field, choose Send to Both. 
This sends the field both the Pass and Fail tables. 


h) Toclose the Rule Editor, choose OK. 
In the If any rule fails and Send to Pass, substitute with: section, select Enabled. 
In the Column field, use the drop-down list to select QUERY .ORDERS ASSIGNED TO. 
In the Expression field, select the ellipsis (...) icon. 


Inthe Smart Editor, enter the expression '3C1la5'. 


Note: 


You must use the single quotation marks before and after the string. 





. Add a validation rule for the shipper's fax to replace any NULL values withNo Fax 


Number. 
a) Inthe input schema area, choose the field SHIPPERFAX . 


b) Inthe Validation Rules area choose Add. 
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c) Enter the name Shipper Fax. 
d) In the Rules area, select the Enabled checkbox if it not already selected. 
e) To open the Rule Editor , select the Column Validation radio box. 
f) In the Column: field, choose Query . SHIPPERFAX . 
g) Inthe Condition: field, choose IS NOT NULL. 
h) In the Action on Fail field, choose Send to Both . 
i) To close the Rule Editor, choose OK . 


j) In the If any rule fails and Send to Pass, substitute with: section, select the check box 
button for the field Enabled . 


k) In the Column field, use the drop-down list to select the field QUERY .SHIPPERFAX. 


1) In the Expression field , select the ellipsis (...) icon and in the Smart Editor, enter the 
expression 'No Fax' and choose OK. and close the editor. 


Note: 


You must use the single quotation marks before and after the string. 





9. Complete the data flow Alpha_Orders_DB DF by connecting the pass, fail, and rule 
violation outputs from the Validation transform to the target template tables. 


a) Return to the data flow workspace. 

b) Select the Validation transform and drag it to the target template table 
Orders DB Work. 
Release the mouse button and choose Pass. 

c) Select the Validation transform and drag it to the target template table 
Orders DB No Fax. 
Release the mouse button and choose Fail. 

d) Select the Validation transform and drag it to the target template table 
Orders DB Rule Violation. 


Release the mouse button and choose Rule Violation. 


10. Execute the Alpha_Orders_Validated_Job and view the differences between passing 
and failing records. 


a) In the Omega project area, right-click onthe Alpha_Orders_Validated_Job and 
choose Execute . 


Data Services prompts you to save any objects that have not been saved. Choose OK. 


b) In the Execution Properties dialog box, choose OK. 
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Note: 
The job should execute successfully, but will cause several errors to appear 
in the Error Log. These errors are records containing values which the 


Designer could not convert because of faulty data. Opening the Error Log 
will display the values which could not be converted. Consequently, these 
records are not moved to the target tables. 





c) To return to the job workspace, choose Back. 
d) To open the Alpha_Orders_DB_ DF data flow workspace, double-click it. 


e) Right click the target tables and choose View data . 
You see the differences between the passing and failing records. 


f) Close the data displays, and return to the job workspace. 
g) Open the Alpha Orders Files DF data flow workspace 


h) Right click the target tables to choose View data. 
You see the differences between the passing and failing records. 


i) Close the data displays. 
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Use the Case Transform 


Business Example 


The Orders data has been validated and merged from two different sources, flat files and 
database tables. Now the resulting data set must be partitioned by quarter for reporting 
purposes. You must use the Case transform to set up the various conditions to partition the 
merged data into the appropriate quarterly partitions 


1. Inthe Omega project, create a new batch job Alpha Orders By Quarter Jobwitha 
new data flow called Alpha Orders By Quarter DF. 


2. Inthe workspace for Alpha_Orders By Quarter DF, add the Orders_Merged table 
from the Alpha datastore as the source object. 


3. Add the Query transform to the data flow workspace after the source table. 
4. Inthe Query Editor , create output columns and map all columns from input to output. 


5. Add the Case transform to the workspace to the right of the Query transform and connect 
them. 


6. In the Case Editor, create the labels and associated expressions for the partitioned fiscal 
quarters 4 in the year 2006 and 1 in the year 2007. 


7. Add three template tables Orders _Q4 2006, Orders Q1 2007,and default output 
to the Delta datastore as output tables for the Case transform and connect them to the 
Case transform. 


8. Execute the Alpha Orders By Quarter Job with the default execution properties. 
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Use the Case Transform 


Business Example 


The Orders data has been validated and merged from two different sources, flat files and 
database tables. Now the resulting data set must be partitioned by quarter for reporting 
purposes. You must use the Case transform to set up the various conditions to partition the 
merged data into the appropriate quarterly partitions 


1. In the Omega project, create a new batch job Alpha_Orders By Quarter Jobwitha 
new data flow called Alpha_Orders By Quarter DF. 


a) Inthe Project area, right-click the Omega project name and choose New Batch Job . 


b) Enter the job name, Alpha_Orders_ By Quarter Job, and, on your keyboard, press 
the Enter key. 


c) To open the job Alpha Orders By Quarter Job workspace, double-click it. 


d) To add the data flow, in the tool palette, choose the Data Flow icon and click in the job 
workspace. 


e) Enter the data flow name, Alpha Orders By Quarter DF and, on your keyboard, 
press the Enter key. 


f) To open the Alpha_Orders_ By Quarter DF workspace, double-click it. 


2. Inthe workspace for Alpha_Orders By Quarter DF, add the Orders_Merged table 
from the Alpha datastore as the source object. 


a) Inthe Local Object Library, choose the Datastores tab. 


b) Expand the Alpha datastore, expand Tables, select the Orders_Merged table, and drag 
it to the workspace. 


3. Add the Query transform to the data flow workspace after the source table. 


a) To add a query to the data flow, in the tool palette, choose the Query Transform icon 
and click inthe Alpha_Orders By Quarter DF workspace. 


b) To connect the source table to the Query , select the source table, hold down the 
mouse button, drag the cursor to the Query and release the mouse button. 


4. Inthe Query Editor , create output columns and map all columns from input to output. 
a) To open the Query Editor, double-click the query. 


b) Inthe Schema In workspace, to select all the fields, select the first field, hold down the 
Shift key, and select the last field. 


c) Drag the selected fields from the Schema In to the Schema Out workspace. 


d) Inthe Schema Out workspace, right-click the last column field and choose New Output 
Column . 
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e) In the dialog box, choose /nsert Below, enter the column name ORDERQUARTER with 
Data Type int, and choose OK. 


f) In the Schema Out workspace, right-click ORDERQUARTER and choose New Output 
Column . 


g) In the dialog box, choose Insert Below , enter the item name ORDERYEAR with Data 
Type varchar (4) and choose OK. 


h) Select the field ORDERQUARTER in the output schema, and, in the Mapping tab, choose 
the Functions button. 


The Select Function dialog box opens. 


i) In the Functions Categories field, choose Date Functions , in the Function name field, 
choose quarter , and choose Next . 


j) In the Define Input Parameters dialog box, select the dropdown arrow to the right of the 
Input date field, and select the Orders_Merged table. 


k) From the table Orders_Merged table, select the ORDERDATE field, choose OK button , 
and, in the next dialog box, choose Finish . 


1) Select the ORDERYEAR field in the output schema, and, in the Mapping tab, choose the 
Functions button. 


m) In the Functions Categories field, choose Conversion Functions , in the Function name 
field, choose to_char and choose Next. 


n) In the Define Input Parameters dialog box, select the dropdown arrow to the right of the 
Input date or number field, and select the Orders_Merged table 


o) From the table Orders_Merged table, select the ORDERDATE field. 


p) In the Format string field, enter 'yyyy' and choose OK. 


Hint: 
e) Remember to put in the single quotation marks before and after the string 


YYYY. 





q) In the next dialog box, choose Finish, and close the editor. 

5. Add the Case transform to the workspace to the right of the Query transform and connect 
them. 
a) In the Local Object Library , choose Transforms tab, and expand the Platform node. 


b) To add the Case transform, select Case and drag it into the data flow workspace. 


c) To connect the Query transform to the Case transform, select the Query transform, 
hold down the mouse button, drag the cursor to the Case transform and release the 
mouse button. 


6. Inthe Case Editor, create the labels and associated expressions for the partitioned fiscal 
quarters 4 in the year 2006 and 1 in the year 2007. 


a) To open the Case Editor , double-click the Case transform to open the transform 
editor. 
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b) To add a new expression, in the Case tab of the Case Editor, choose the Add button. 
c) Inthe Label field, enter the label 942006 for the expression. 


d) In the input schema, select the ORDERQUARTER column, drag it to the Expression 
workspace at the bottom of the window and type = 4 and. 


e) To complete the expression for the first condition, in the input schema, select the 
ORDERYEAR column, drag it to the Expression workspace at the bottom of the window 
and type ='2006' . 


The expression should appear as: 
Query .ORDERQUARTER = 4 and Query.ORDERYEAR = '2006' 


f) Repeat steps b to e for the following expressions: 


Q12007 Query .ORDERQUARTER = 1 and 
Query .ORDERYEAR = '2007' 





g) Todirect records that do not meet any defined conditions to a separate target object, 
confirm that the Produce default output with label checkbox is selected, and that the 
label name default is entered in the associated field. 


h) To direct records that might meet multiple conditions to only one target, confirm that 
the Row can be TRUE for one case only checkbox is selected. 


In this case, records are placed in the target associated with the first condition that 
evaluates as true. 


i) Return to the data flow workspace. 


7. Add three template tables Orders _Q4 2006, Orders Q1 2007, anddefault_output 
to the Delta datastore as output tables for the Case transform and connect them to the 
Case transform. 


a) To add a new template table to the data flow, in the tool palette, choose the Template 
Table icon and click in the workspace. 


b) Inthe Create Template dialog box, in the Table Name field, enter Orders _Q4 2006. 


c) Inthe In datastore drop-down list, choose the Delta datastore as the template table 
destination target and choose OK. 


d) Repeat steps a to c for the next five tables, using the following data: 


default_output default_output 





e) Connect the output from the Case transform to the target template tables. 
Repeat this step for each of the template tables. 


8. Execute the Alpha Orders By Quarter Job with the default execution properties. 
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a) In the Omega project area, right-click the Alpha_Orders By Quarter Job and 
choose Execute . 


Data Services prompts you to save any objects that have not been saved. 


b) Inthe Save all changes and execute dialog box, choose Yes. 
The Execution Properties dialog box appears. 


c) To execute the job using the default execution properties, in the Execution Properties 
dialog box, choose OK. 


d) Return to the job workspace. 
e) Open the data flow workspace 


f) Right click the target table Orders Q1_ 2007 and choose View Data . 
Note that the titles for the affected contacts are changed. 


g) Confirm that there are 90 orders that were placed in fiscal quarter one of 2007. 


h) Close the data display. 
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Use the SQL Transform 


Business Example 


Your company extracts data from external systems using flat files. The data volume from the 
various external systems has increased continually in the recent past, making management of 
the jobs for flat file extraction difficult. You can optimize this process by using Data Services 
to extract data directly from an external system. 


You use the SQL transform to submit SQL commands to generate data to be moved into 
target objects where other transforms do not meet business requirements. 


The contents of the Employee and Department tables must be merged, so you use the SQL 
transform to merge the tables. 


1. In the Omega project, create a new batch job called Alpha_Employees Dept Job 
containing a data flow called Alpha_Employees Dept DF. 


2. Add an SQL transform to the data flow and connect it to the Emp_Dept table from the 
HR_datamart datastore as the target object. 


3. In the transform editor for the SQL transform, specify the source datastore and tables. 


4. Execute the Alpha Employees Dept Job with the default execution properties. 
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Use the SQL Transform 


Business Example 


Your company extracts data from external systems using flat files. The data volume from the 
various external systems has increased continually in the recent past, making management of 
the jobs for flat file extraction difficult. You can optimize this process by using Data Services 
to extract data directly from an external system. 


You use the SQL transform to submit SQL commands to generate data to be moved into 
target objects where other transforms do not meet business requirements. 


The contents of the Employee and Department tables must be merged, so you use the SQL 
transform to merge the tables. 


1. Inthe Omega project, create a new batch job called Alpha_Employees Dept Job 
containing a data flow called Alpha_Employees Dept DF. 


a) In the Project area, right-click the project name and choose New Batch Job . 


b) Enter the job name Alpha_Employees_ Dept _Job, and, on your keyboard, press the 
Enter key. 


c) To open the job Alpha Employees Dept Job, double-click it. 


d) To add a new data flow to the Alpha Employees Dept Job, in the tool palette, 
choose the Data Flow icon and click in the workspace. 


e) Enter the data flow name Alpha Employees Dept_DF, and, on your keyboard, press 
the Enter key. 


f) To open the data flow workspace, double-click the data flow. 


2. Add an SQL transform to the data flow and connect it to the Emp_Dept table from the 
HR_datamart datastore as the target object. 


a) Inthe Local Object Library , select the Transforms tab, and expand the Platform node. 
b) To add the SQL transform, select SQL and drag it to the data flow workspace. 


c) Inthe Local Object Library, select the Datastores tab and expand the HR_Datamart 
datastore. 


d) Select the EMP_DEPT table, drag it to the data flow workspace, and choose Make 
Target . 


3. Inthe transform editor for the SQL transform, specify the source datastore and tables. 
a) To open the Transform Editor , double-click the SQL transform. 


b) On the SQL tab, in the field Datastore field, use the drop-down list to choose the Alpha 
datastore. 


c) In the Database type field, use the drop-down list to choose Sybase ASE 15.X. 
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d) Inthe SQL text workspace, enter the following expression: 


SELECT employee.EMPLOYEEID, employee.FIRSTNAME, 
employee.LASTNAME, department . DEPARTMENTNAME 


FROM employee, department 
WHERE employee.DEPARTMENTID = department .DEPARTMENTID 


This SQL statement selects the last name and first name of the employee from the 
Employee table, and the department to which the employee belongs. It looks up the 
value in the Department table based on the Department ID. 


e) Tocreate the output schema, choose the Update schema button. 
This creates the output column fields. 


f) Right-click the EMPLOYEEID column and choose Primary Key. 
g) Close the editor. 


h) To connect the SQL transform to the target table, select the SQL transform, hold down 
the mouse button, drag the cursor to the target table, and release the mouse button. 


4. Execute the Alpha_Employees_ Dept _Job with the default execution properties. 


a) Inthe Omega project area, right-click the Alpha_Employees Dept _Job and choose 
Execute. 
Data Services prompts you to save any objects that have not been saved. 


b) In the Save all changes and execute dialog box, choose Yes. 
The Execution Properties dialog box appears. 


c) To execute the job using default properties, choose OK. 
d) Go Back to the job workspace. 
e) To open the data flow workspace, double click the data flow. 


f) Right click the target table and choose View data. 


You should have 40 rows in your target table, because there were 8 employees in the 
Employee table with department IDs that were not defined in the Department table . 


g) Close the display. 
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Create an Alternative Work Flow 


Business Example 


With the influx of new employees resulting from Alpha's acquisition of new companies, the 
Employee Department information needs to be updated regularly. Because this information is 
used for payroll, it is critical that there is no loss of records if a job is interrupted. You need to 
set up the job in a way that exceptions are always managed. This involves setting up a 
conditional that executes a less resource-intensive update of the table first. If that generates 
an exception, the conditional then tries a version of the same data flow that is configured to 
auto correct the load. 


Set up a job Alpha_Empoyees Dept Recovery Job with a try/catch block and conditional 
to catch exceptions in the execution of a data flow Alpha Employees Dept_DF. Exceptions 
cause the conditional to execute a different version of the same data flow 

Alpha Employees Dept AC DF configured with auto correction. 


1. Replicate the data flow Alpha_Employees Dept DF as 
Alpha Employees Dept AC DFinthe Local Object Library and reconfigure the target 
table in Alpha_Employee_Dept_AC_DF for auto correction. 


2. Inthe Omega project, create a new batch job called 
Alpha_Employees Dept Recovery Job and anew global variable 
$G Recovery Needed. 


3. Inthe workspace of the Alpha_Employees Dept Recovery Job add a work flow called 
Alpha_Employees Dept Recovery WF. 


4. Inthe Alpha_Employees Dept Recovery WF workspace, add a script called 
GetStatus and construct an expression to update the value of the global variable 
$G_Recovery Needed to the same value as in the recovery_flag column in the 
recovery_status table in the HR_datamart. 


5. In the work flow workspace, enter a Conditional called Alpha_Employees Dept Con 
connected to the script. 


6. Configure the Conditional as an “if” statement that determines which data flow to execute 
based upon the value of the global variable $G Recovery Needed. 


7. Execute Alpha_Employees Dept Recovery Job with the default properties. 
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Create an Alternative Work Flow 


Business Example 


With the influx of new employees resulting from Alpha's acquisition of new companies, the 
Employee Department information needs to be updated regularly. Because this information is 
used for payroll, it is critical that there is no loss of records if a job is interrupted. You need to 
set up the job in a way that exceptions are always managed. This involves setting up a 
conditional that executes a less resource-intensive update of the table first. If that generates 
an exception, the conditional then tries a version of the same data flow that is configured to 
auto correct the load. 


Set up a job Alpha_Empoyees Dept Recovery Job With a try/catch block and conditional 
to catch exceptions in the execution of a data flow Alpha Employees Dept_DF. Exceptions 
cause the conditional to execute a different version of the same data flow 

Alpha Employees Dept_AC_DF configured with auto correction. 


1. Replicate the data flow Alpha_Employees Dept DF as 
Alpha Employees Dept AC DFinthe Local Object Library and reconfigure the target 
table in Alpha_Employee_Dept_AC_DF for auto correction. 


a) Inthe Local Object Library, select the Data Flows tab, right-click the 
Alpha_Employees Dept_DF data flow and choose Replicate . 


b) To change the name of the replicated data flow to Alpha Employees Dept AC DF, 
right-click the data flow, choose Rename, enter the new name, and, on your keyboard, 
press the Enter key. 


c) ToopenAlpha_ Employees Dept _AC DF, inthe Local Object Library , select the 
Data Flows tab, and double-click the Alpha Employees Dept AC DF data flow. 


d) To open the Target Table Editor , double-click the target table Emp_Dept . 
e) Inthe Target Table Editor , select the Options tab. 

f) Change the value in the Auto correct load field from No to Yes. 

g) Go Back to the data flow workspace. 


2. Inthe Omega project, create a new batch job called 
Alpha_Employees Dept Recovery Job and anew global variable 
$G Recovery Needed. 


a) Inthe project area, right-click the Omega project, choose New batch job, and enter the 
job name Alpha_Employees Dept Recovery Job. 


b) In the project area, select the job Alpha_Employees Dept Recovery Job, and, 
from the main menu, choose Tools — Variables . 


c) Right-click Global Variables and choose Insert . 
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d) Right-click the new global variable, choose Properties , and, in the Global Variable 
Properties dialog box, in the Name field , enter $G_ Recovery Needed. 


e) In the Data type dropdown list, choose int and choose OK. 
f) Close the Variables and Parameters editor. 
3. Inthe workspace of the Alpha Employees Dept Recovery Job add a work flow called 
Alpha_Employees Dept Recovery WF. 
a) In the tool palette, select the Work Flow icon, click in the job workspace, and enter the 


name Alpha Employees Dept Recovery WF. 


b) To open the workflow workspace, double-click 
Alpha_Employees Dept Recovery WF. 


4. Inthe Alpha_Employees Dept Recovery WF workspace, add a script called 
GetStatus and construct an expression to update the value of the global variable 
$G_Recovery Needed to the same value as in the recovery_flag column in the 
recovery_status table in the HR_datamart. 


a) To add a script to the Alpha_Employees Dept_Recovery WF workspace, in the tool 
palette, choose the Script icon, and click in the workspace. 

b) Name the script GetStatus. 

c) To open the script, double-click it. 


d) Type in the following expression: 


$G_ Recovery Needed = sql('HR Datamart','select RECOVERY FLAG from 
RECOVERY_STATUS') ; 


This expression updates the value of the global variable to the value as in the 
recovery_flag column in the recovery_status table in the HR_datamart 
e) Close the script and go Back to the work flow workspace. 
5. In the work flow workspace, enter a Conditional called Alpha_Employees Dept Con 
connected to the script. 
a) In the tool palette, choose the Conditional icon, and click in the work flow workspace. 


b) Enter the name of the Conditional Alpha_Employees_ Dept _Con and, on your 
keyboard, press the Enter key. 


c) To connect the script and the conditional, select the script, hold down the mouse 
button, drag it to the Conditional and release the mouse button. 


d) To open the Conditional Editor , double-click the Conditional . 


6. Configure the Conditional as an “if” statement that determines which data flow to execute 
based upon the value of the global variable $G_ Recovery Needed. 


a) Inthe Alpha_Employees Dept_Con Conditional Editor , in the “if” statement field, 
enter the expression: $G Recovery Needed = 0. 


Note: 


Do not type the period at the end of this expression. 
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This “if” statement states that recovery is not required. 


b) Inthe tool palette, choose the Try icon, click in the “Then” area of the Conditional 
Editor and enter the Try name Alpha_Employees Dept Try. 


c) Inthe Local Object Library, select the data flow Alpha_Employees Dept_DF and 
drag the data flow into the “Then” pane of the Conditional Editor , 


d) Connect the Alpha_Employees Dept _ Tryo Alpha Employees Dept _DF. 


e) Inthe tool palette, choose the Catch icon, click in the “Then” pane of the Conditional 
Editor , and enter the name Alpha_Employees Dept Catch. 


f) Connect the Alpha_Employees Dept Catch to Alpha Employees Dept DF. 
g) To open the Catch Editor , double click Alpha_Employees Dept Catch. 


h) To add a script to the catch, in the tool palette, choose the Script icon, click in the 
lower paneCatch Editor workspace, enter the script name Recovery Fail and, on 
your keyboard, press the Enter key. 


i) Double-click the Recovery_Fail script and enter the following expression: 
sql('HR Datamart','update RECOVERY STATUS set RECOVERY FLAG = 
1'); 
This expression updates the flag in the recovery status table to 1, indicating that 
recovery is needed. 


j) Close the Script. 


k) In the Local Object Library, select the Data Flows tab, select 
Alpha Employees Dept _AC_DFand drag it to the “Else” pane of the Conditional 
Editor . 


1) In the tool palette, choose the Script icon, click in the “Else” pane of the Conditional 
Editor to the right of the data flow, and enter the script name Recovery Pass. 


m) Double-click the Recovery_Pass script and enter the expression: 
sql('HR Datamart', 'update RECOVERY STATUS set RECOVERY FLAG = 
0"); 
This expression updates the flag in the recovery status table to O, indicating that 
recovery is not needed to update the flag in the recovery status table to O, indicating 
that recovery is not needed. 


n) Close the Script. 


0) ConnectAlpha Employees Dept AC DFtothe script Recovery Pass. The script 
should be downstream from the data flow. 
7. Execute Alpha_Employees Dept Recovery Job with the default properties. 


a) Inthe project area, select Alpha_Employees Dept Recovery Job and choose 
Execute . 


Data Services prompts you to save any objects that have not been saved. 


b) In the Save all changes and execute dialog box, choose Yes . 
The Execution Properties dialog box appears. 


66 © Copyright. All rights reserved. 


Solution 14: Create an Alternative Work Flow 


c) To execute the job using default properties, choose OK 


Note: 
The trace log indicates the data flow generated an error, but the job 


completed successfully because of the try catch block. An error log that 
indicates a primary key conflict in the target table was generated. 





d) Execute the Alpha_Employees Dept Recovery Joba second time with the default 
properties. 


e) In the Project area of the designer, select the Log tab at the bottom. 


f) Open the node for Alpha_Employees_Dept_Recovery_Job and select the last entry of 
successful execution in the resulting list. 


g) In the Log view in the workspace to the right of the Project Area, select the first 
DATAFLOW log entry and note that the dataflow used was 
Alpha_Employees_Dept_AC_DF. 
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Use Source-Based Change Data Capture (CDC) 


Business Example 


You need to set up a job to update employee records in the Omega data warehouse whenever 
they change. The employee records include time stamps to indicate when they were last 


updated, so you can use source-based CDC. 


Construct and configure a batch job Alpha_Employees_ Job, which updates employee table 
columns based on whether records are new or have been changed since the last time data 


was updated. 


1. In the Omega project, create a new batch job and data flow called 
Alpha_Employees Dim Job and a new global variable $G LastUpdate. 


2. Inthe job Alpha Employees Dim Job Workspace, add a script called GetTimeStamp 
and construct an expression to select the last time the job executed and on that basis, if 
the time stamp is NULL, then all records are processed. If the time stamp is not NULL, 
then assign the value to the global variable $G_LastUpdate. 


3. Inthe job Alpha Employees Dim Job workspace, add a data flow 
Alpha Employees Dim DF to the right of the script and connect it to the script. 


4. Add the Employee table from the Alpha datastore as the source object and the EMP_DIM 
table from the Omega datastore as the target object of the data flow 
Alpha Employees Dim DF. Connect them with a Query transform. 


5. Map the Schema In fields of the Query transform to the Schema Out fields, as follows: 


Schema In 


Schema Out 





EMPLOYEEID 
LASTNAME 
FIRSTNAME 
BIRTHDATE 
HIREDATE 
ADDRESS 
CITYID 
REGIONID 
COUNTRYID 
PHONE 
EMAIL 
REPORTSTO 
DEPARTMENTID 


EMPLOYEEID 
LASTNAME 
FIRSTNAME 
BIRTHDATE 
HIREDATE 
ADDRESS 
CITY 

REGION 
COUNTRY 
PHONE 
EMAIL 
REPORTSTO 
DEPARTMENT 
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Schema In Schema Out 


LastUpdate LAST_UPDATE 


DISCHARGE_DATE DISCHARGE_DATE 





6. Create a mapping expression for the SURR_KEY column that generates new keys based 
on the EMP_DIM target table incrementing by 1 by using the Functions wizard. 


7. Onthe WHERE tab, construct an expression to select only those records with a time 
stamp that is later than the value of the global variable $G LastUpdate. 


8. View the data in the source and target tables before executing the job. 


9. Execute Alpha Employees Dim Job with the default properties. 
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Use Source-Based Change Data Capture (CDC) 


Business Example 


You need to set up a job to update employee records in the Omega data warehouse whenever 
they change. The employee records include time stamps to indicate when they were last 
updated, so you can use source-based CDC. 


Construct and configure a batch job Alpha_Employees_ Job, which updates employee table 
columns based on whether records are new or have been changed since the last time data 
was updated. 


1. In the Omega project, create a new batch job and data flow called 


Alpha_Employees Dim Job and a new global variable $G LastUpdate. 


a) Inthe project area, right-click the Omega project, choose New batch job and enter the 
name Alpha Employees Dim Job. 


b) Select the job Alpha_Employees Dim Job and, in the main menu, choose 
Tools — Variables . 


c) Right-click Global Variables and choose Insert . 
d) Right-click the new variable, and choose Properties. 


e) In the Global Variable Properties dialog box, enter the name $G_LastUpdate, enter 
the Data type datetime and choose OK. 


f) Close the Variables and Parameters window. 


. Inthe job Alpha Employees Dim Job workspace, add a script called GetTimeStamp 


and construct an expression to select the last time the job executed and on that basis, if 
the time stamp is NULL, then all records are processed. If the time stamp is not NULL, 
then assign the value to the global variable $G_LastUpdate. 


a) To add the script to the Alpha Employees Dim Job workspace, in the tool palette, 
choose the Script icon, click in the workspace, and enter the name Get TimeStamp . 


b) To open the GetTimeStamp script, double-click it. 


c) Inthe script, enter the following expression: 
$G_LastUpdate = to_date(sql('Omega', 'select max (LAST UPDATE) 
from EMP DIM'), 'MON DD YYYY HH:MI') ; 
if ($G LastUpdate is null) $G LastUpdate = to date ('1901.01.01', 
"YYYY.MM.DD') ; 
else print('Last update was ' || $G_LastUpdate) ; 


This expression updates the value of the global variable to the value of the last update 
column in the employee dimension table. The script: 
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a) Selects the last time the job was executed from the last update column in the 
employee dimension table. 


b) If the last update column is NULL, assigns a value of January 1, 1901 to the 
$G_LastUpdate global variable. When the job executes for the initial load, this 
ensures that all records are processed. 


c) If the last update column is not NULL, assign the actual time stamp value to the 
$G_LastUpdate global variable, and print the value of the variable to the job's log 
file. 


d) Close the Script and go Back to the job workspace. 


3. Inthe job Alpha Employees Dim Job workspace, add a data flow 
Alpha Employees Dim DF to the right of the script and connect it to the script. 
a) To add the data flow, in the tool palette, choose the Data Flow icon, click in the job 
workspace and enter the data flow name, Alpha_Employees Dim DF. 


b) To connect the GetTimeStamp to Alpha_Employees Dim DF data flow, select the 
script, hold down the mouse button, drag the cursor to the data flow and release the 
mouse button. 


c) To open the data flow workspace, double-click Alpha_Employees Dim DF. 


4. Add the Employee table from the Alpha datastore as the source object and the EMP_DIM 
table from the Omega datastore as the target object of the data flow 
Alpha Employees Dim DF. Connect them with a Query transform. 


a) Inthe Local Object Library, select the Datastores tab. 


b) From the Alpha datastore, select the Employee table, drag it into the data flow 
workspace and choose Make Source . 


c) From the Omega datastore, select the EMP_DIM table, drag it into the data flow 
workspace and choose Make Target . 


d) To add the query, in the tool palette, choose the Query Transform icon and click in the 
data flow workspace. 


e) Connect the source table to the query and connect the query to the target table. 


5. Map the Schema In fields of the Query transform to the Schema Out fields, as follows: 





Schema In Schema Out 
EMPLOYEEID EMPLOYEEID 
LASTNAME LASTNAME 
FIRSTNAME FIRSTNAME 
BIRTHDATE BIRTHDATE 
HIREDATE HIREDATE 
ADDRESS ADDRESS 
CITYID CITY 
REGIONID REGION 
COUNTRYID COUNTRY 
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Schema In Schema Out 


PHONE PHONE 
EMAIL EMAIL 


REPORTSTO REPORTSTO 


DEPARTMENTID DEPARTMENT 
LastUpdate LAST_UPDATE 
DISCHARGE_DATE DISCHARGE_DATE 





a) To open the Query Editor , double-click the query. 


b) To map the columns in the Schema In pane to the columns in the Schema Out pane, 
select the column and drag it from Schema In to Schema Out. 


. Create a mapping expression for the SURR_KEY column that generates new keys based 


on the EMP_DIM target table incrementing by 1 by using the Functions wizard. 
a) Inthe Schema Out pane, choose the SURR_KEY column. 


b) In the Mapping tab, choose the Function button. 


c) Inthe Functions Categories field, choose Database Functions, in the Function Name 
field, choose the Key_generation function, and choose Next. 


d) Inthe Define Input Parameters dialog box, enter the parameters: 
Field/Option 


Table Omega .dbo.EMP DIM 


Key_column SURR_KEY 





Key_increment 1 


e) Choose Finish . 


You see the expression key_generation('Omega.dbo.EMP DIM', 'SURR_KEY', 
1). 


. On the WHERE tab, construct an expression to select only those records with a time 


stamp that is later than the value of the global variable $G_LastUpdate. 
a) Inthe Query Editor , select the WHERE tab. 


b) Inthe workspace, enter the following expression: 
employee.LastUpdate > $G LastUpdate 


c) Close the editor. 


. View the data in the source and target tables before executing the job. 


a) Inthe Data Flow workspace, choose the View Data icon on the employee source table. 
b) Choose the View Data icon on the EMP_DIM target table. 
c) Note the number of rows in each table. 


d) Close both View Data windows. 
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9. Execute Alpha Employees Dim Job with the default properties. 


a) In the project area, select the Alpha Employees Dim Job and choose Execute . 
b) To save all the objects that you have created, choose Yes . 


c) To execute the job using default properties, choose OK. 
According to the log, the last update for the table was on “2007.10.04” 


d) Return to the job workspace. 

e) To open the data flow workspace, double-click the data flow. 
f) Right-click the target table and choose View data . 

g) Sort the records by the LAST_UPDATE column. 

h) Close the display. 
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Business Example 


You find that some of your data does not provide any time stamps or logs to provide a source- 
based CDC. You want to investigate using target-based CDC to compare the source to the 
target to determine which records have changed. 


You need to set up a job to update product records in the Omega data warehouse to capture 
change. The product records do not include time stamps to indicate when they were last 
updated. Use target-based change data capture to extract all records from the source and 
compare them to the target 


1. In the Omega project, create a new batch job called Alpha_Product_Dim_Job containing 
a data flow called Alpha Product Dim DF. 


2. Inthe workspace for Alpha Product Dim DF,add the Product table from the Alpha 
datastore as the source object and the Product _Dim table from the Omega datastore as 
the target object. 


3. Add a Query transform to the workspace connecting it to the source and target objects. In 
addition, add the Table Comparison, History Preserving and Key Generation transforms to 
the workspace. 


4. In the transform editor for the Query transform, map input columns to output columns. by 
dragging corresponding columns from the input schema to the output schema. After 
deleting the link between the Query transform and the target table, complete the 
connection of the remaining objects in the data flow workspace. 


5. In the transform editor for the Table Comparison transform, use the PRODUCT_DIM table 
in the Omega datastore as the comparison table and set the field SURR_KEY as the 
generated key column. 


6. Inthe transform editor for the History Perserving transform, use COST as the compare 
columns to preserve history for records having cost changes. 


7. Inthe transform editor for the Key Generation transform, set up key generation based on 
the SURR_KEY column of the PRODUCT_DIM table and increment the key by a value of 1. 


8. In the data flow workspace, before executing the job, display the data in both the source 
and target tables. 


9. Execute the Alpha Product Dim Job with the default execution properties. 
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Business Example 


You find that some of your data does not provide any time stamps or logs to provide a source- 
based CDC. You want to investigate using target-based CDC to compare the source to the 
target to determine which records have changed. 


You need to set up a job to update product records in the Omega data warehouse to capture 
change. The product records do not include time stamps to indicate when they were last 
updated. Use target-based change data capture to extract all records from the source and 
compare them to the target 


1. Inthe Omega project, create a new batch job called Alpha Product Dim Job containing 
a data flow called Alpha Product Dim DF. 


a) In the Project area, right-click the Omega project name and choose New Batch Job . 


b) Enter the job name, Alpha Product Dim Job and, on your keyboard, press the 
Enter key. 


Ifthe Alpha_Product_Dim_ Job does not open automatically, to open, double-click 
the job. 


c) To add the data flow to the Alpha Product Dim Job, in the tool palette, choose the 
Data Flow icon, and click in the workspace, enter the name Alpha_Product_Dim_ DF 
and, on your keyboard, press the Enter key. 


d) To open the data flow workpspace, double-click Alpha Product Dim DF. 


2. Inthe workspace for Alpha_Product_Dim_DF, add the Product table from the Alpha 
datastore as the source object and the Product_Dim table from the Omega datastore as 
the target object. 


a) Inthe Local Object Library, choose the Datastores tab. 


b) In the Alpha datastore, select the Product table, drag it to the data flow workspace, 
and choose Make Source . 


c) In the Omega datastore, select the PRODUCT_DIM table, drag it to the data flow 
workspace, and choose Make Target . 


3. Add a Query transform to the workspace connecting it to the source and target objects. In 
addition, add the Table Comparison, History Preserving and Key Generation transforms to 
the workspace. 


a) To add the query to the Alpha_Product_Dim_DF, in the tool palette, choose the 
Query Transform icon and click in the workspace. 


b) Connect the source table, Product, to the Query transform. 


c) Connect the target table PRODUCT_DIM to the Query transform. 
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d) Inthe Local Object Library , choose the Transforms tab and expand the Data Integrator 
node. 


e) Select Table Comparison and drag it to the data flow workspace to the bottom of the 
dataflow workspace. 


f) Select History Preserving and drag it to the data flow workspace to the right of the 
Table Comparison transform. 


g) Select Key Generation and drag it to the data flow workspace to the right of the History 
Preserving transform. 


. Inthe transform editor for the Query transform, map input columns to output columns. by 


dragging corresponding columns from the input schema to the output schema. After 
deleting the link between the Query transform and the target table, complete the 
connection of the remaining objects in the data flow workspace. 


a) Double-click the Query transform to open the Query Editor . 


b) Inthe Schema In workspace select the following fields, and drag them to the 
corresponding fields to the Schema Out workspace. 


Schema In Schema Out 


PRODUCTID PRODUCTID 
PRODUCTNAME PRODUCTNAME 
CATEGORYID CATEGORYID 
COST COST 





c) Select the output schema field SURR_KEY and, on the Mapping tab, enter the value 
NULL. 


This provides a value until a key can be generated. 
d) Select the output schema field EFFECTIVE_DATE and, on the Mapping tab, enter the 
value sysdate( ). 


Do not add a period to the end of this expression. 


This provides the system current date as the effective date. 
e) Close the editor. 


f) To delete the link between the Query transform and the target table, right-click the link 
and choose Delete . 


g) Toconnect the Query transform to the Table Comparison transform, click the Query 
transform, hold down the mouse button, drag the cursor to the Table Comparison 
transform and release the mouse button. 


h) Repeat the above step to connect the following: 
The Table Comparison transform and the History Preserving transform. 


The History Preserving transform and the Key Generation transform. 
The Key Generation transform and the target table. 


. Inthe transform editor for the Table Comparison transform, use the PRODUCT_DIM table 


in the Omega datastore as the comparison table and set the field SURR_KEY as the 
generated key column. 
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a) To open the Transform Editor , double-click the Table Comparison transform. 


b) Inthe Table Comparison tab, use the drop-down list for the Table name filed, and select 
PRODUCT_DIM in the Omega datastore as the comparison table from which the 
maximum existing key is determined. 

The PRODUCT_DIM is the comparison table from which the maximum existing key is 
determined. 


c) Use the drop-down list for the Generated key column field, and select SURR_KEY as the 
generated key column. 


d) Inthe Schema In, select the PRODUCTNAME , CATEGORYID and COST fields and drag 
these to the Compare columns field. 


e) In the Schema In, select the field PRODUCTID field and drag it to the Input primary key 
column(s) field. 


f) Close the editor. 


6. In the transform editor for the History Perserving transform, use COST as the compare 
columns to preserve history for records having cost changes. 
a) To open the History Preserving Transform Editor, double-click the History Preserving 
transform. 
b) From the list of fields in the Schema In, drag COST to Compare columns. 
c) Close the editor. 
7. Inthe transform editor for the Key Generation transform, set up key generation based on 
the SURR_KEY column of the PRODUCT_DIM table and increment the key by a value of 1. 
a) To open the Key Generation Transform Editor , double-click the Key Generation 
transform. 


b) In the drop-down list for the Table name field select PRODUCT_DIMin the Omega 
datastore. 


The PRODUCT_DIM is the comparison table from which the maximum existing key is 
determined. 


c) In the drop-down list for the Generated key column field, select SURR_KEY as the 
generated key column. 


d) In the Increment Value field, enter 1. 
e) Close the editor. 


8. In the data flow workspace, before executing the job, display the data in both the source 
and target tables. 


a) In the data flow workspace, select the magnifying glass on the source table. 
A large View Data pane appears beneath the current workspace. 


b) In the data flow workspace, select the magnifying glass on the target table. 
A large View Data pane appears beneath the current workspace. 


c) Note that the “OmegaSoft” product has been added in the source, but has not yet 
been updated in the target. 


9. Execute the Alpha Product Dim Job with the default execution properties. 
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a) Inthe Omega project area, right-click on the Alpha_Product_Dim_Job and choose 
Execute . 


b) In the Save all changes and execute dialog box, choose Yes. 


c) To execute the job using default properties, in the Execution Properties dialog box, 
choose OK. 


d) Return to the job workspace. 
e) To open the data flow workspace, double-click the data flow. 


f) Right click the target table and choose View data . 


Note that there are new records for “product IDs 2, 8, 13" and that “OmegaSoft” has 
been added to the target. 


g) Close the display. 
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Use the Data Transfer Transform 


Task 1: Join Database Schemas 

The data transfer transform can be used to push data down to a database table, so that it can 
be processed by the database server rather than the data services job server. In this activity, 
join two database schemas. When the data transfer transform is not used, the join occurs on 
the data services job server. 


1. Inthe Omega project, create a new batch job called Alpha Data Transfer Job, 
containing a data flow called Alpha Data Transfer DF. 


2. Inthe Alpha_Data_Transfer_DF workspace, add the following as source objects: 
- The Employee Comp table from the Delta datastore. 
- The Employee table from the Alpha datastore. 

3. Add a query transform to the workspace and connect each source object to it. 


4. Inthe transform editor for the Query transform, add the FirstName, LastName, and 
BirthDate columns from the Employee table and the Comp_Type and Comp columns from 
the Employee_Comp table to the output schema. Join the two tables on the EmployeelD 
columns. 


Caution: 


Create a target template table Employee_Temp in the Delta datastore. Then 
save the batch job. 





5. Display the optimized SQL for the data flow. 


Task 2: Use Data Transfer Transform and Query Transform 

Modify the new batch job Alpha Data Transfer Job containing the data flow 

Alpha Data Transfer DF and use the Data Transfer transform in addition to the Query 
transform. 


1. Open the dataflowAlpha Data Transfer Job. 


2. Add a Data Transfer transform to the workspace and place it between the source table 
Employee and the Query transform. 


3. Configure the Data Transfer transform to push the join of data to the database server. 


4. Configure the Query transform to join the Data Transfer transform output to the source 
table input. Save the objects and execute the job. 


5. Display the optimized SQL for the data flow. 
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Use the Data Transfer Transform 


Task 1: Join Database Schemas 

The data transfer transform can be used to push data down to a database table, so that it can 
be processed by the database server rather than the data services job server. In this activity, 
join two database schemas. When the data transfer transform is not used, the join occurs on 
the data services job server. 


1. In the Omega project, create a new batch job called Alpha Data Transfer Job, 
containing a data flow called Alpha Data Transfer DF. 


a) In the Project area, in the project context menu, choose New Batch Job. 
b) Enter the name ofthe job: Alpha Data Transfer Job. 
c) Press Enter to commit the change. 
d) Double-click the A/pha_Data_Transfer_Job job to open it. 
e) Inthe Tool Palette, select the Data Flow icon. 
f) Select the workspace to which you want to add the data flow. 
g) Enter Alpha Data Transfer DFasthe name. 
h) Press Enter to commit the change. 
i) Double-click the data flow to open the data flow workspace. 
2. Inthe Alpha_Data_Transfer_ DF workspace, add the following as source objects: 
+ The Employee Comp table from the Delta datastore. 
e The Employee table from the Alpha datastore. 
a) Inthe Local Object Library, select the Datastores tab. 
b) In the Delta datastore, select the Employee Comp table. 
c) Inthe Local Object Library, select the Datastores tab. 
d) Inthe Alpha datastore, select the Employee table and choose Make Source. 


3. Add a query transform to the workspace and connect each source object to it. 
a) Inthe Tool Palette, choose the Query transform icon. 
Select the workspace to which you want to add the query template. 


b) To connect the source table Employee Comp to the Query transform, select the 
source table and, holding down the mouse button, drag the cursor to the Query 
transform. Then release the mouse button to create the connection. 
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c) To connect the source table Employee to the Query transform, select the source table 
and, holding down the mouse button, drag the cursor to the Query transform. Then 
release the mouse button to create the connection. 


4. Inthe transform editor for the Query transform, add the FirstName, LastName, and 
BirthDate columns from the Employee table and the Comp_Type and Comp columns from 
the Employee_Comp table to the output schema. Join the two tables on the EmployeelD 
columns. 


Caution: 


Create a target template table Employee_Temp in the Delta datastore. Then 
save the batch job. 





a) To open the Query transform editor, double-click the Query transform. 


b) From the Schema In workspace, drag the fields FirstName, LastName and BirthDate 
columns from the Employee table to the Schema Out workspace. 


Hint: 
Q If you drag and drop each Schema In column to the root node of the 
Query, the output column will automatically go to the bottom of the list. 





c) From the Schema In workspace, drag the fields Comp_Type and Comp columns from 
the Employee_Comp table to the Schema Out workspace. 


Hint: 
Q If you drag and drop each Schema In column to the root node of the 


Query, the output column will automatically go to the bottom of the list. 





d) In the FROM tab, you should see both source tables, EMPLOYEE and 
EMPLOYEE_COMP in the Input Schema(s) section. 


e) In the Join Pairs section, use the drop-down box for the Left field to select the source 
table EMPLOYEE. 


f) In the Join Type field, use the drop down box to select the value Inner join. 
g) In the Right field, use the drop down box to select the source table EMPLOYEE COMP. 


h) From the Input Schema, drag the EMPLOYEEID field from the EMPLOYEE table into the 
Join Condition field. 


Note: 


If the drag and drop does not work, then use the Smart Editor by clicking 
the button with the three dots. 





i) In the Join Condition field, type an equal sign =. 
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j) From the Input Schema, drag the field EMPLOYEE ID from the EMPLOYEE_COMP table 
into the field Join Condition. 


The proposed expression should be: 


EMPLOYEE . EMPLOYEEID = EMPLOYEE COMP .EMPLOYEEID 
k) Toclose the editor, choose the Back icon. 


1) To add a new template table to the data flow, in the Tool Palette, select the Template 
Table icon and select the workspace where you want to add it. 


m) Inthe Create Template dialog box, enter Employee Temp as the template table name. 


n) Inthe /n datastore drop down list, select the Delta datastore as the template table 
destination target. 


o) Choose OK. 


p) To connect the Query transform to the target template table Employee Temp, select 
the Query transform and, holding down the mouse button, drag the cursor to the 
template table and release the mouse button. 


q) To save all objects that you have created, on the Designer tool bar, choose Save All. 


5. Display the optimized SQL for the data flow. 


a) In the Local Object Library area, navigate to the data flow tab and, in the context menu 
of the Alpha Data Transfer Job job, choose Display Optimized SQL. 


b) Select the first datastore on the left to view the SQL that this data flow applies against 
the corresponding database. 


Note the absence of a Join statement in the SQL. 

c) Select the second datastore on the left to view the SQL that this data flow applies 
against the corresponding database. 
Note the absence of a Join statement in the SQL. 


Task 2: Use Data Transfer Transform and Query Transform 

Modify the new batch job Alpha Data Transfer Job containing the data flow 

Alpha Data Transfer DF and use the Data Transfer transform in addition to the Query 
transform. 


1. Open the dataflow Alpha_Data_Transfer Job. 


2. Add a Data Transfer transform to the workspace and place it between the source table 


Employee and the Query transform. 


a) From the context menu of the link between the source table Employee and the Query 
transform, choose Delete. 


b) Inthe Local Object Library, select the Transforms tab. 


c) From the Data Integrator node, select the Data Transfer transform and drag it to the 
data flow workspace. 


d) To connect the source table Employee to the Data Transfer transform, select the 
source table and, holding down the mouse button, drag the cursor to the Data Transfer 
transform. Then release the mouse button to create the connection. 
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e) To connect the Data Transfer transform to the Query transform, select the Data 
Transfer transform and, holding down the mouse button, drag the cursor to the Query 
transform. Then release the mouse button to create the connection. 


3. Configure the Data Transfer transform to push the join of data to the database server. 
a) To open the Data Transfer transform editor, double-click the Data Transfer transform. 


b) In the Transfer Type field, select the option Table. 

c) In the Table Options section of the transform editor, select the elipses (...) button. 
d) Select the Delta datastore and select Table Name. 

e) Inthe Table Name field, enter PUSHDOWN_DATA with DBO in the Owner field. 

f) To close the editor, choose Back. 


4. Configure the Query transform to join the Data Transfer transform output to the source 
table input. Save the objects and execute the job. 
a) To open the Query transform editor, double-click the Query transform. 
b) On the FROM tab, to update the expression to join on the EMPLOYEEID fields in the 


EMPLOYEE_COMP and DATA_TRANSFER sources, highlight the existing expression 
and delete it. 


The expression on the FROM tab should look like this: 
DATA_TRANSFER.EMPLOYEEID = EMPLOYEE COMP .EMPLOYEEID 


c) Verify that the fields Comp_Type and Comp columns are mapped to the 
Employee_Comp source. 
To do this, select the output columns Comp_Type and Comp in succession to see if the 
mapped field comes from the Employee_Comp part of the Schema In pane. 

d) Toclose the editor, choose Back. 


e) To save all objects that you have created, on the Designer tool bar, choose Save All . 


5. Display the optimized SQL for the data flow. 
a) Inthe Local Object Library area, navigate to the data flow tab and in the context menu 
of the Alpha_Data_Transfer_ DF, choose Display Optimized SQL. 
b) Select the datastore on the left to view the SQL that this data flow applies against the 
corresponding database. 
Note the presence of a Join statement in the SQL. 
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